Image processing device and program

ABSTRACT

An object of the present invention is to provide an image processing technology capable of perceiving a motion of the shape of a target area. An image processing device ( 3 ) according to the present invention includes: a dynamic image acquiring unit ( 110 ) that acquires a dynamic image; a boundary line extracting means ( 130 ) that acquires a plurality of target area boundary lines by extracting boundary lines of a target area; a displacement correcting means ( 140 ) that acquires a predetermined number of displacement-corrected boundary lines in which a removal-required component is removed by calculating a displacement amount, which is the removal-required component, using a base boundary line as a displacement base for one or more of target area boundary lines other than the base boundary line among the plurality of target area boundary lines by using pixels corresponding to the plurality of target area boundary lines and correcting a predetermined number of the target area boundary lines other than the base boundary line by using the displacement amount after the calculation of the displacement amount; and a display means ( 34 ) that displays displacement-corrected boundary line information for display based on the predetermined number of displacement-corrected boundary lines.

CROSS REFERENCE TO RELATED APPLICATION

This Application is a 371 of PCT/JP2014/060287 filed on Apr. 9, 2014which, in turn, claimed the priority of Japanese Patent Application No.JP2013-104336 filed on May 16, 2013, both applications are incorporatedherein by reference.

TECHNICAL FIELD

The present invention relates to an image processing technology of adynamic image acquired by photographing a human body or an animal body.

BACKGROUND ART

At medical sites, by photographing an affected part included in aninternal organ, a skeleton, or the like by using an X ray or the like,various tests and diagnoses are made. In recent years, by applyingdigital technologies, a dynamic image (an image group configured by aplurality of frame images) in which a motion of an affected part isacquired by using an X ray or the like can be acquired relativelyeasily.

Thus, since a dynamic image of a subject area including a diagnosistarget area can be acquired by using a semiconductor image sensor suchas a flat panel detector (FPD), a pathological analysis and a diagnosisbased on a motion analysis of a diagnosis target area or the like, whichcannot be performed according to conventional still-image photographingand analysis using X-ray photographing, have been attempted.Particularly, in a dynamic-state analysis of a chest X-ray, supports fora diagnosis/treatment (CAD for an X-ray dynamic image) using afunctional and quantitative analysis of a dynamic-state relating to achange in the density of the inside of a pulmonary field for eachposition inside the pulmonary field have been also reviewed.

As a method for the quantitative analysis described above, a technologyhas been proposed in which analysis information effective for adiagnosis is generated by analyzing a temporal change based on frameimages of a dynamic-state image of the chest.

For example, in a technology disclosed in Patent Literature 1, thetechnology has been disclosed which generates a new image bycontinuously acquiring a plurality of X-ray images in a time series,setting a line at a desired position for each of the plurality of X-rayimages, acquiring a pixel row aligned along the set line, and aligningacquired pixel rows in the order of a time series.

In addition, in a technology disclosed in Patent Literature 2, thetechnology for acquiring a moving amount by measuring the position ofthe diaphragm based on a dynamic image, acquiring relative ventilationinformation for each divided chest area by specifying a dynamic image atthe time of maximal inhalation and a dynamic image at the time ofmaximal exhalation and using a pixel differential value, executinglinear interpolation between CT images, generating a coronal image, asagittal image, and a Raysum image, measuring the position of thediaphragm based on the Raysum image, executing position adjustmentbetween frames of the dynamic images of which the respiratory levelsmatch those of the CT images and the Raysum image generated based on theCT images, and overlapping the ventilation information with the coronalimage and the dynamic image has been disclosed. In addition, a methodfor measuring the positions of the horizontal diaphragm and a pulmonaryapex, acquiring a moving amount, and displaying the graph of the motionbased on the moving amount has been disclosed.

CITATION LIST Patent Literatures

Patent Literature 1: JP 2004-312434 A

Patent Literature 2: WO 2006/137294 A

SUMMARY OF INVENTION Technical Problem

However, while a diagnosis image represented using the method disclosedin Patent Literature 1 represents a change in the time direction in afixed line as one cross-sectional image, the motion of the shape of atarget area in the time direction (in other words, a temporal change ofa two-dimensional space on a frame image) cannot be represented.

Meanwhile, also in Patent Literature 2, while the method for measuringspecific positions of the horizontal diaphragm and the pulmonary apex,acquiring moving amounts (displacement amounts) at the positions, anddisplaying the motions at the positions based on the moving amounts hasbeen disclosed, and a motion of the target area can be represented inone dimension, the motion of the two-dimensional shape of the targetarea cannot be represented.

In other words, according to the technologies disclosed in PatentLiteratures 1 and 2, the temporal change of the two-dimensional space onthe frame image cannot be acquired, and thus, the motion of the shape ofthe target area cannot be acquired.

The present invention is in consideration of such situations, and anobject thereof is to provide an image processing technology capable ofacquiring the motion of the shape of a target area desired by a user.

Solution to Problem

In order to solve the above problem, an image processing device which isan invention of claim 1 includes: a dynamic image acquiring means thatacquires a dynamic image configured by a plurality of frame imagesacquired by sequentially photographing a time-varying physical state ofa target area inside a human body or an animal body in a time direction;a boundary line extracting means that executes a boundary lineextracting process in which a plurality of target area boundary linesare acquired by extracting boundary lines of the target area for aplurality of frame images among the plurality of frame images; adisplacement correcting means that acquires a predetermined number ofdisplacement-corrected boundary lines in which a removal-requiredcomponent is removed by executing a displacement amount calculatingprocess in which a displacement amount, which is the removal-requiredcomponent, is calculated using a base boundary line as a displacementbase for one or more of target area boundary lines other than the baseboundary line among the plurality of target area boundary lines by usingpixels corresponding to the plurality of target area boundary lines andexecuting a correction process in which a predetermined number of thetarget area boundary lines other than the base boundary line arecorrected by using the displacement amount after the displacement amountcalculating process; and a display means that displaysdisplacement-corrected boundary line information for display based onthe predetermined number of displacement-corrected boundary lines.

Further, an invention of claim 2 is the image processing deviceaccording to claim 1, wherein the removal-required component includes atleast one component among deformation components according to a verticalmotion, a parallel motion, and rotation in the target area.

Further, an invention of claim 3 is the image processing deviceaccording to claim 1 or 2, further including a frame selecting meansthat executes a frame selecting process including a process of selectinga base frame image used for extracting the base boundary line and areference frame image used for extracting the target area boundary linesother than the base boundary line for selection target frame imagesincluding at least the plurality of frame images, wherein thedisplacement amount calculating process includes a process ofcalculating a displacement amount between corresponding pixels of thetarget area boundary line of the base frame image as the base boundaryline and the target area boundary line of the reference frame image.

Further, an invention of claim 4 is the image processing deviceaccording to claim 3, wherein the selection target frame images includeframe images photographed in the past in time with respect to theplurality of frame images, and the frame selecting process includes aprocess of selecting a frame image photographed for the same body in thepast in time with respect to the plurality of frame images as the baseframe image.

Further, an invention of claim 5 is the image processing deviceaccording to claim 3, further including a period classifying means thatdetects a target area period in which a periodical change of the targetarea of the body synchronized with photographing time at which theplurality of frame images are photographed occurs and classifies theplurality of frame images in units of the target area periods, whereinthe base frame image and the reference frame image are frame images whenthe target area periods are within a same period, a value representing atime-varying physical state of the target area is defined as a physicalstate value, and the frame selecting process includes a first selectionprocess selecting one frame image as the base frame image from among(b1) a frame image when the physical state value corresponds to a firstset value set in advance, (b2) a frame image when the physical statevalue corresponds to a maximum value, and (b3) a frame image when thephysical state value corresponds to a minimum value, and a secondselection process selecting one frame image as the reference frame imagefrom among (c1) a frame image when the physical state value correspondsto a second set value set in advance, (c2) a frame image that isadjacent to the base frame image in time, (c3) a frame imagecorresponding to the minimum value of the physical state value when thebase frame image is the frame image of (b2), and (c4) a frame imagecorresponding to the maximum value of the physical state value when thebase frame image is the frame image of (b3).

Further, an invention of claim 6 is the image processing deviceaccording to claim 3, wherein the displacement amount used for thecorrection process is a displacement amount between corresponding pixelsof the base boundary line and one of the target area boundary lines, andthe target area boundary lines other than one of the target areaboundary lines are corrected using the displacement amount.

Further, an invention of claim 7 is the image processing deviceaccording to claim 3, wherein the displacement amount used for thecorrection process is a displacement amount from the target areaboundary line nearest in time from the target area boundary line that isa correction target.

Further, an invention of claim 8 is the image processing deviceaccording to claim 3, wherein the displacement amount used for thecorrection process is a displacement amount between the base boundaryline and the target area boundary line, which is the correction target,that is acquired as a sum of displacement amounts of two boundary linesadjacent in time.

Further, an invention of claim 9 is the image processing deviceaccording to any one of claims 1 to 8, further including an imagegenerating means that generates a predetermined number of separateimages separated for each predetermined number of displacement-correctedboundary lines, wherein the display means sequentially displays thepredetermined number of separate images as displacement-correctedboundary line information.

An invention of claim 10 is the image processing device according to anyone of claims 1 to 9, further including an image generating means thatgenerates one still image such that the plurality ofdisplacement-corrected boundary lines are displayed in an overlappingmanner, wherein the display means displays the still image as thedisplacement-corrected boundary line information.

Further, an invention of claim 11 is the image processing deviceaccording to any one of claims 1 to 10, wherein the target area includesat least one of a diaphragm area and a heart area.

Further, an invention of claim 12 is a program that causes a computer toserve as the image processing device according to any one of claims 1 to11 by being executed by the computer included in the image processingdevice.

Advantageous Effects of Invention

According to an image processing device disclosed in claims 1 to 11, adisplacement amount calculating process is executed in which adisplacement amount is calculated using a base boundary line as adisplacement base for one or more of target area boundary lines otherthan a base boundary line among a plurality of target area boundarylines by using pixels corresponding to the plurality of target areaboundary lines, and the displacement amount is a removal-requiredcomponent. Then, a correction process is executed in which apredetermined number of the target area boundary lines other than thebase boundary line are corrected by using the displacement amount afterthe displacement amount calculating process, whereby a predeterminednumber of displacement-corrected boundary lines in which aremoval-required component is removed are acquired.Displacement-corrected boundary line information for display based onthe predetermined number of displacement-corrected boundary lines isdisplayed. In other words, by displaying the displacement-correctedboundary lines acquired by removing the deformation corresponding to thedisplacement amount from the target area boundary lines, the userperceives a change in the shape of the target area boundary line,whereby the motion of the shape of the target area can be perceived. Inaddition, since the shape of the target area boundary line can beperceived, a partial abnormality of the shape can be found, and thepartial abnormality such as adhesion can be easily diagnosed.Furthermore, since the diagnosis content desired by the user iscollected in the displacement-corrected boundary line information, aminimum requisite diagnosis time is necessary, whereby the efficiency ofthe diagnosis is improved. For this reason, a dynamic-state diagnosiscan be executed appropriately and efficiently.

According to the image processing device of claim 2, theremoval-required component includes at least one component amongdeformation components according to a vertical motion, a parallelmotion, and rotation in the target area. Accordingly, thedisplacement-corrected boundary lines acquired by removing thedeformations due to the vertical motion, the parallel motion, and therotation from the target area boundary lines can be displayed.

According to the image processing device of claim 3, the frame selectingmeans is further included which executes, for selection target frameimages including at least a plurality of frame images, the frameselecting process including the process of selecting the base frameimage used for extracting the base boundary lines and reference frameimages used for extracting the target area boundary lines other than thebase boundary line, and the displacement calculating process includesthe process of calculating a displacement amount between correspondingpixels of the target area boundary line of the base frame image as thebase boundary line and the target area boundary lines of the referenceframe image. Accordingly, frame images corresponding to user's diagnosispurpose can be selected, and the displacement-corrected boundary lineinformation corresponding to the diagnosis purpose can be displayed. Inaddition, by selecting only necessary frame images, compared to the casewhere the displacement-corrected boundary lines are acquired for all theframe images included in the dynamic image, a calculation time requiredfor the boundary line extracting process, the displacement amountcalculating process, and the correction process can be minimized.

According to the image processing device of claim 4, the selectiontarget frame images include frame images photographed in the past intime with respect to the plurality of frame images, and the frameselecting process includes a process of selecting a frame imagephotographed for the same body in the past in time with respect to theplurality of frame images as the base frame image. In other words, in acase where the present displacement-corrected boundary line informationindicating the present displacement-corrected boundary lines isacquired, the base frame image can be executed by using a common (same)frame image photographed in the past. Accordingly, through adynamic-state diagnosis, a comparison between the past and the presentin the diaphragm boundary line LI of one body and a comparison betweenchanges thereof can be made with high accuracy. For this reason, afollow-up observation can be accurately executed.

According to the image processing device of claim 5, the base frameimage and the reference frame image are frame images when the targetarea periods are within a same period, and the frame selecting processincludes a first selection process selecting one frame image as the baseframe image from among (b1) to (b3) and a second selection processselecting one frame image as the reference frame image from among: (c1)to (c4). Accordingly, between frame images within the same perioddesired by the user, a change of the shape of the target area boundaryline can be diagnosed with high precision.

According to the image processing device of claim 6, the target areaboundary lines other than one of the target area boundary lines can becorrected with high accuracy by using the displacement amount betweencorresponding pixels of the base boundary line and one target areaboundary line.

According to the image processing device of claim 7, the displacementamount used for the correction process is a displacement amount from thetarget area boundary line nearest in time from the target area boundaryline that is a correction target. In other words, the reference frameimage is changed for each calculation of the displacement amount, andthe base frame image can be also changed. In this way, the base frameimage can be changed in accordance with the change of the referenceframe image.

Thus, by executing the correction process constantly using thedisplacement amount between diaphragm boundary lines of the selectiontarget frame images that are nearest, the displacement-correctedboundary lines having higher correction accuracy can be acquired. As aresult, the displacement-corrected boundary line information that issuitable for the diagnosis purpose can be displayed, and accordingly,the dynamic-state diagnosis can be executed more appropriately andefficiently.

According to the image processing device of claim 8, the displacementamount used for the correction process is a displacement amount betweenthe base line boundary line and the target area boundary line, which isthe correction target, that is acquired as a sum of displacement amountsof two boundary lines adjacent in time. In this way, in the displacementamount calculating process, one displacement amount between the baseboundary line and the diaphragm boundary line is subdivided andcalculated, and accordingly, compared to the displacement amountcalculated without subdividing the base frame image to the referenceframe image, the displacement amount can be calculated with highaccuracy.

According to the image processing device of claim 9, a predeterminednumber of separate images separated for each predetermined number ofdisplacement-corrected boundary lines are generated, and thepredetermined number of separate images are sequentially displayed asdisplacement-corrected boundary line information. Thus, a change of theshape of the target area boundary line can be perceived based on thedynamic image.

According to the image processing device of claim 10, one still image isgenerated such that a predetermined number of displacement-correctedboundary lines are displayed in an overlapping manner, and the stillimage is displayed as the displacement-corrected boundary lineinformation. For example, in a case where the shape of the predeterminednumber of the displacement-corrected boundary lines, which is adiagnosis target, is set as the displacement-corrected boundary lineinformation, the displacement-corrected boundary lines can be displayedto be identifiable in an overlapping manner. Accordingly, a change ofthe shape of the target area boundary line can be perceived on one stillimage.

According to the image processing device of claim 11, the target areaincludes at least one of a diaphragm area and a heart area. Accordingly,diseases such as pulmonary emphysema and diaphragmatic eventration canbe appropriately diagnosed through a dynamic-state diagnosis. In such adisease, in the case of a minor symptom, while the abnormality may notbe noticed, by making a diagnosis using the displacement-correctedboundary line information, the diagnosis does not depend on the user'ssubjectivity, whereby an erroneous diagnosis can be prevented.

According to the image processing device of claim 12, the same effectsas those of the inventions of claims 1 to 11 can be acquired.

Objects, features, aspects, and advantages of the present invention willbecome more apparent by detailed description presented below and theattached drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram that illustrates the whole configuration of aradiation dynamic image photographing system 100 according to a firstembodiment.

FIG. 2 is a diagram that illustrates a relation between a respiratorymotion and the position of the diaphragm.

FIG. 3 is a block diagram that illustrates the functional configurationof an image processing device 3 according to the first embodiment.

FIG. 4 is a diagram that illustrates a dynamic image acquired throughradiation dynamic image photographing as an example.

FIG. 5 is a schematic view that illustrates a respiration phaserepresenting waveform data of a respiration vibration value in a timeseries and photographing timings altogether.

FIG. 6 is a schematic view that illustrates a diaphragm boundary line atan exhalation phase.

FIG. 7 is a schematic view that illustrates a diaphragm boundary line atthe time of an exhalation phase as an example with coordinates beingassigned thereto.

FIG. 8 is a schematic view that illustrates extraction of the contour ofa pulmonary field area including a diaphragm boundary line as anexample.

FIG. 9 is a schematic view that illustrates extraction of the contour ofthe pulmonary field area including the diaphragm boundary line as anexample.

FIG. 10 is a schematic view that illustrates a displacement amountbetween diaphragm boundary lines.

FIG. 11 is a schematic diaphragm that illustrates a displacement amountbetween diaphragm boundary lines.

FIG. 12 is a schematic view that illustrates displacement-correctedboundary lines at the time of an exhalation phase as an example.

FIG. 13 is a schematic view that illustrates a displacement amountcalculating process.

FIG. 14 is a schematic view that illustrates a displacement amountcalculating process.

FIG. 15 is a schematic view that illustrates a correction process.

FIG. 16 is a schematic view that illustrates a correction process.

FIG. 17 is a schematic view that illustrates a correction process.

FIG. 18 is a diagram that illustrates displacement-corrected boundaryline information for a healthy person as an example.

FIG. 19 is a diagram that illustrates displacement-corrected boundaryline information for an unhealthy person as an example.

FIG. 20 is a flowchart that illustrates the basic operation of an imageprocessing device 3 realized according to the first embodiment.

FIG. 21 is a schematic view that illustrates a correction processaccording to a second embodiment.

FIG. 22 is a schematic view that illustrates a correction processaccording to a third embodiment.

FIG. 23 is a block diagram that illustrates the functional configurationof an image processing device 3A according to a fourth embodiment.

FIG. 24 is a diagram that illustrates a method of calculating arespiration period.

FIG. 25 is a flowchart that illustrates the basic operation of an imageprocessing device 3A realized in a fourth embodiment.

FIG. 26 is a block diagram that illustrates the functional configurationof an image processing device 3B according to a fifth embodiment.

FIG. 27 is a diagram that illustrates a frame selecting processaccording to the fifth embodiment.

FIG. 28 is a flowchart that illustrates the basic operation of an imageprocessing device 3B realized according to a fifth embodiment.

FIG. 29 is a schematic view that illustrates heart boundary linesaccording to a sixth embodiment.

FIG. 30 is a schematic view of a heartbeat phase representing the motionof a heart wall in a time series.

FIG. 31 is a diagram that illustrates an example of a method ofdisplaying displacement-corrected boundary line information.

DESCRIPTION OF EMBODIMENTS 1. First Embodiment

Hereinafter, a radiation dynamic image photographing system according toa first embodiment of the present invention will be described.

<1-1. Whole Configuration of Radiation Dynamic Image PhotographingSystem>

The radiation dynamic image photographing system according to the firstembodiment photographs a radiation image for a situation in which thephysical state of a target area of a subject periodically changes intime by using a human body or an animal body as the subject.

FIG. 1 is a diagram that illustrates the whole configuration of theradiation dynamic image photographing system 100 according to the firstembodiment. As illustrated in FIG. 1, the radiation dynamic imagephotographing system 100 includes: a photographing apparatus 1; aphotographing control device 2 (photographing console); and an imageprocessing device 3 (diagnosing console). The photographing apparatus 1and the photographing control device 2 are interconnected through acommunication cable or the like, and the photographing control device 2and the image processing device 3 are configured to be interconnectedthrough a communication network NT such as a local area network (LAN).Each device configuring the radiation dynamic image photographing system100 is compliant with the digital image and communications in medicine(DICOM) standard, and communication between the devices is executed incompliance with the DICOM standard.

<1-1-1. Configuration of Photographing Apparatus 1>

The photographing apparatus 1, for example, is an apparatus that isconfigured by an X-ray photographing apparatus or the like andphotographs the dynamic state of the chest of a subject M according torespiration. The photographing of a dynamic state is executed byacquiring a plurality of images in order of time while repeatedlyemitting a radiation ray such as an X ray to the chest of the subject M.A series of images acquired by such continuous photographing are calleda dynamic image. In addition, each of a plurality of images configuringthe dynamic image is called a frame image.

As illustrated in FIG. 1, the photographing apparatus 1 is configured toinclude: a radiation unit (radiation ray source) 11; a radiationexposure control device 12; an imaging unit (radiation detecting unit)13, and a reading control device 14.

The radiation unit 11 emits a radiation ray (X ray) to the subject M inaccordance with control of the radiation exposure control device 12. Anexample illustrated in the figure is a system for a human body, and thesubject M corresponds to an inspection target person. Hereinafter, thesubject M may be also referred to as an “examinee”.

The radiation exposure control device 12 is connected to thephotographing control device 2 and executes radiation photographing bycontrolling the radiation unit 11 based on a radiation exposurecondition input from the photographing control device 2.

The imaging unit 13 is configured by a semiconductor image sensor suchas an FPD and converts a radiation ray that is emitted from theradiation unit 11 and is transmitted through the examinee M into anelectrical signal (image information).

The reading control device 14 is connected to the photographing controldevice 2. The reading control device 14 acquires image data by switchingthe reading of an electrical signal stored in each pixel and reading theelectrical signal stored in the imaging unit 13 by controlling theswitching unit of each pixel of the imaging unit 13 based on an imagereading condition input from the photographing control device 2. Then,the reading control device 14 outputs the acquired image data (frameimage) to the photographing control device 2. The image readingcondition, for example, is a frame rate, a frame interval, a pixel size,an image size (matrix size), or the like. The frame rate is the numberof frame images acquired per second and matches a pulse rate. The frameinterval is a time from the start of operation of acquiring a frameimage of one time to the start of operation of acquiring a next frameimage in continuous photographing and matches a pulse interval.

Here, the radiation exposure control device 12 and the reading controldevice 14 are interconnected, exchange a synchronization signal, and areconfigured to synchronize a radiation exposure operation and an imagereading operation with each other.

<1-1-2. Configuration of Photographing Control Device 2>

The photographing control device 2 controls the radiation photographingoperation and the operation of reading a radiation image executed by thephotographing apparatus 1 by outputting a radiation control conditionand an image reading condition to the photographing apparatus 1 anddisplays a dynamic image acquired by the photographing apparatus 1 forchecking whether it is an image appropriate for checking or diagnosingpositioning executed by a cameraman.

As illustrated in FIG. 1, the photographing control device 2 isconfigured to include a control unit 21, a storage unit 22, an operationunit 23, a display unit 24, and a communication unit 25, and such unitsare interconnected through a bus 26.

The control unit 21 is configured by a central processing unit (CPU),random access memory (RAM), and the like. The CPU of the control unit 21reads a system program or various processing programs stored in thestorage unit 22 and expands the program within the RAM in accordancewith operation using the operation unit 23, executes various operationstarting from a photographing control process to be described lateraccording to the expanded program, thereby controlling the operation ofeach unit of the photographing control device 2 and the operation of thephotographing apparatus 1 in a centralized manner.

The storage unit 22 is configured by nonvolatile semiconductor memory, ahard disk, or the like. The storage unit 22 stores various programsexecuted by the control unit 21, parameters necessary for the executionof the process for the programs, or data such as a processing result.

The operation unit 23 is configured by a keyboard including a cursorkey, numeric input keys, various functional keys, and the like and apointing device such as a mouse and outputs an instruction signal inputthrough a key operation for the keyboard, a mouse operation, or a touchpanel to the control unit 21.

The display unit 24 is configured by a monitor such as a color liquidcrystal display (LCD) and displays an input instruction supplied fromthe operation unit 23, data, or the like in accordance with aninstruction for a display signal input from the control unit 21.

The communication unit 25 includes a LAN adapter, a modem, a terminaladapter (TA) or the like and controls transmission/reception of datato/from each device connected to the communication network NT.

<1-1-3. Configuration of Image Processing Device 3>

The image processing device 3 acquires a dynamic image transmitted fromthe photographing apparatus 1 through the photographing control device 2and displays an image used for radiogram interpretation to be made by adoctor or the like.

As illustrated in FIG. 1, the image processing device 3 is configured toinclude a control unit 31, a storage unit 32, an operation unit 33, adisplay unit 34, a communication unit 35, and an analysis unit 36, andsuch units are interconnected through a bus 37.

The control unit 31 is configured by a CPU, RAM, and the like. The CPUof the control unit 31 reads a system program or various processingprograms stored in the storage unit 32 in accordance with operationusing the operation unit 33, expands the program into the RAM, andexecutes various processes according to the expanded program, therebycontrolling the operation of each unit of the image processing device 3in a centralized manner (detailed description will be presented later).

The storage unit 32 is configured by nonvolatile semiconductor memory, ahard disk, or the like. The storage unit 32 stores various programsexecuted by the control unit 31, parameters necessary for the executionof the process for the programs, or data such as a processing result.For example, the storage unit 32 stores an image processing program usedfor executing image processing to be described later. Such variousprograms are stored in the form of a readable program code, and thecontrol unit 31 sequentially executes operation according to the programcode.

The operation unit 33 is configured by a keyboard including a cursorkey, numeric input keys, various functional keys, and the like and apointing device such as a mouse and outputs an instruction signal inputthrough a key operation for the keyboard, a mouse operation, or a touchpanel to the control unit 31.

The display unit 34 is configured by a monitor such as a color LCD anddisplays an input instruction supplied from the operation unit 33, data,and a display image to be described later in accordance with aninstruction for a display signal input from the control unit 31.

The communication unit 35 includes a LAN adapter, a modem, a TA or thelike and controls transmission/reception of data to/from each deviceconnected to the communication network NT.

<1-2. Relation Between Respiratory Motion and Position of Diaphragm andProblem in Diagnosis of Dynamic State>

As a premise for detailed description of the image processing device 3according to this embodiment, a relation between a respiratory motionand the position of the diaphragm and problems in the diagnosis of adynamic state according thereto will be described.

FIG. 2 is a diagram that illustrates a general relation between arespiratory motion and the position of the diaphragm. FIG. 2A is aschematic view of the side face of the inside of a human body at thetime of breathing in (at the time of inhalation), FIG. 2B is a schematicview of the side face of the inside of the human body at the time ofbreathing out (at the time of exhalation), and FIG. 2C is a schematicview that represents situations on the side face of the inside of thehuman body both at the time of exhalation and at the time of inhalationaltogether.

As illustrated in FIG. 2C when left and right thoracic cavities 52 thatare sealed by being enclosed by the thorax 53 and the diaphragm 50inflate, the air is inhaled, and, when the left and right thoraciccavities 52 contract, the air is exhaled, whereby respiration is made.In other words, as illustrated in FIG. 2A, at the time of inhalation,the diaphragm 50-1 is lowered as denoted by arrow AR11, and a rib islifted as denoted by arrow AR12. Accordingly, the thoracic cavities 52expand, and the air is inhaled into the lung through a trachea 51 asdenoted by arrow AR13. On the other hand, as illustrated in FIG. 2B, atthe time of exhalation, the diaphragm 50-2 is lifted as denoted by arrowAR21, and the rib is lowered as denoted by arrow AR22. Accordingly, thethoracic cavities 52 are narrowed, and the air is exhaled from the lungthrough the trachea 51 as denoted by arrow AR23. Such a motion of thediaphragm 50 takes charge of about 60% of the respiratory motion.

In the example illustrated in FIG. 2 described above, while the relationbetween the respiratory motion of a healthy person and the position ofthe diaphragm 50 has been described, for example, for an unhealthyperson who has suffered from illness such as pulmonary emphysema, abronchus becomes thin due to a damage of pulmonary alveoli, and, evenwhen a breath is given out, the air does not come out from the lung, thelung excessively expands, and a state is maintained in which the lungpresses down the diaphragm 50 in the state in which a breath is givenout. In other words, the diaphragm 50 is not moved, and the respiratorymotion cannot be made well.

In addition, for an unhealthy person who has suffered from illness ofthe diaphragm such as diaphragmatic eventration, in a state in which oneside or both sides of the diaphragm 50 are not moved, a phrenic nervemoving the diaphragm 50 is disordered caused by mediastinum, tumors ofthe lung, an aortic aneurysm, an external wound, operation of themediastinum, or the like, whereby paralysis of the diaphragm occurs.

In making a dynamic-state diagnosis for an unhealthy person who hassuffered from such illness, when an inspection is made by using a chestX-ray dynamic image, a state is formed in which the diaphragm 50 doesnot move even when respiration is made with being pressed down or raisedup.

While it is possible even for a user such as an experienced doctor toeasily make a diagnosis in a case where the illness is serious, in thecase of minor illness, the diagnosis depends on user's subjectivity,ambiguity may remain, which may lead to making a wrong diagnosis.

In addition, in order to prevent the wrong diagnosis described above,the user needs to make a dynamic-state diagnosis by making a detailedobservation or making an observation of a difference from the motionaccording to a healthy person's respiration. Thus, a lot of observationtime is consumed, whereby the diagnosis efficiency is very low.

Furthermore, since the diaphragm has a cubic shape, it is difficult toperceive the motion of the diaphragm by perceiving the shape of thediaphragm based on an X-ray dynamic image that has been actuallyphotographed. In addition, it is necessary to make a diagnosis based onthe shape of left and right pulmonary fields and a change in the motion.However, for a user who has insufficient experiences of diagnosesaccording to a dynamic-state diagnosis, it is difficult to perceive anddetermine an abnormal shape change in the left and right pulmonaryfields based on a dynamic image in which a plurality of motions such asa positional variation and a shape variation are caused even in a normalrespiration operation.

Under such a background, it is desirable to display analysis informationthat is effective for a diagnosis without depending on the user'ssubjectivity.

Thus, in the present invention, by displaying boundary line informationacquired by removing unnecessary deformations between frame images fromthe boundary line of a target area, it is possible to perceive themotion of the shape of a target area.

Hereinafter, the image processing device 3 according to the firstembodiment will be described in detail.

<1-3. Specific Configuration of Image Processing Device 3>

The image processing device 3 of the radiation dynamic imagephotographing system 100 according to the first embodiment of thepresent invention displays boundary line information acquired byremoving deformations according to a vertical motion, a parallel motion,and rotation between frame images, whereby a dynamic-state diagnosis canbe made appropriately and efficiently.

Hereinafter, the functional configuration realized by the imageprocessing device 3 will be described.

<1-3-1. Functional Configuration of Image Processing Device 3>

FIG. 3 is a diagram that illustrates the functional configurationrealized by the control unit 31 by operating the CPU and the likeaccording to various programs in the image processing device 3 of theradiation dynamic image photographing system 100 together with otherconfigurations. The image processing device 3 according to thisembodiment uses a dynamic image acquired by photographing the chestmainly including the heart and both lungs.

The control unit 31 is mainly configured by: a dynamic-image acquiringunit 110; a frame selecting unit 120; a boundary line extracting unit130; a displacement correcting unit 140; and an image generating unit150.

Hereinafter, while the functional configuration of the control unit 31as illustrated in FIG. 3 will be described to be realized by executing aprogram that has been installed in advance, the functional configurationmay be realized by a dedicated hardware configuration.

Hereinafter, specific contents of the processes executed by the dynamicimage acquiring unit 110, the frame selecting unit 120, the boundaryline extracting unit 130, the displacement correcting unit 140, and theimage generating unit 150 will be sequentially described with referenceto FIG. 3.

<1-3-1-1. Dynamic Image Acquiring Unit 110>

The dynamic image acquiring unit 110 acquires a dynamic image configuredby a plurality of frame images acquired by sequentially photographing inthe time direction changing states of the physical state of a targetarea with respect to time inside the body of an examinee M that arephotographed by the reading control device 14 of the photographingapparatus 1. The target area according to this embodiment is assumed tobe a diaphragm area. In other words, as illustrated in FIG. 3, thephotographing control device 2 is interposed between the photographingapparatus 1 and the image processing device 3, and detection data (aplurality of frame images MI) stored in the storage unit 22 of thephotographing control device 2 is output to the communication unit 35 ofthe image processing device 3 through the communication unit 25.

FIG. 4 is a diagram that illustrates a dynamic image acquired throughradiation dynamic image photographing of dynamic states of the chest ofthe examinee M accompanied with respiration as an example. Asillustrated in FIG. 4, frame images M1 to M10 (MI) acquired by thedynamic image acquiring unit 110 are acquired by continuouslyphotographing for one period of the respiratory cycle at predeterminedphotographing timings. More specifically, images acquired at thephotographing timings of time t=t1, t2, t3, . . . , t10 respectivelycorrespond to frame images M1, M2, M3, . . . , M10.

<1-3-1-2. Frame Selecting Unit 120>

The frame selecting unit 120 executes a frame selecting processincluding the process of selecting a base frame image BF used forextracting a base boundary line BL to be described later and a referenceframe image RF used for extracting diaphragm boundary lines LI (to bedescribed later in detail) other than the base boundary line BL forselection target frame images TI including at least a first number of (aplurality of) frame images MI that is at least two or more.

FIG. 5 is a schematic view that illustrates a respiration phase PHrepresenting waveform data of a (physical state value (respirationvibration value) to be described later in a time series andphotographing timings TM altogether and is a diagram that illustratesthe frame selecting process. As illustrated in FIG. 5, for example, in acase where the selection target frame images TI are assumed to be T1 toT5, in the frame selecting process, the base frame image BF and thereference frame image RF are selected from among the frame images T1 toT5 (TI).

<1-3-1-3. Boundary Line Extracting Unit 130>

The boundary line extracting unit 130 executes a boundary lineextracting process in which a first number of (a plurality of) targetarea boundary lines are acquired by extracting boundary lines of atarget area from the selection target frame image TI. Since the targetarea according to this embodiment is the diaphragm area, hereinafter,the target area boundary lines will be described as diaphragm boundarylines.

The boundary line extracting unit 130 executes the boundary lineextracting process for the base frame image BF and the reference frameimage RF selected by the frame selecting unit 120. In other words, thediaphragm boundary line LI for the base frame image BF corresponds tothe base boundary line BL, and the diaphragm boundary line LI for thereference frame image RF corresponds to the reference boundary line RL.Then, the boundary line extracting unit 130 outputs the base boundaryline BL and the reference boundary line RL to the displacementcorrecting unit 140 (see FIG. 3).

FIGS. 6 and 7 are schematic views that illustrate diaphragm boundarylines at the time of an exhalation phase PH2 (see FIG. 5). FIG. 7 is aschematic view that illustrates the diaphragm boundary lines in anorthogonal coordinate system at the time of the exhalation phase PH2 inan area R1 illustrated in FIG. 6.

As illustrated in FIGS. 6 and 7, diaphragm boundary lines L1 to L5 (LI)are respectively extracted from the selection target frame images T1 toT5 (TI) illustrated in FIG. 5 described above that correspond to thebase frame image BF and the reference frame image RF.

Hereinafter, while a method of extracting diaphragm boundary lines willbe described more specifically, the extraction method is not limited tothe method described here, but any method capable of extractingdiaphragm boundary lines from a dynamic image may be used.

<1-3-1-3-1. First Boundary Line Extracting Process>

A first boundary line extracting process is a process of extracting adiaphragm boundary line LI by executing contour extraction of apulmonary field part based on the selection target frame image TI. FIG.8 is a schematic view that illustrates contour extraction of a pulmonaryfield part including the diaphragm boundary line LI. The pulmonary fieldpart, as illustrated in FIG. 8, may be extracted for each of the leftand right sides (see FIG. FIG. 8a ) or may be extracted as a contour(see FIG. 8B) including the areas of the heart and the spine. As thisextraction method, a conventional technology (for example, see “Imagefeature analysis and computer-aided diagnosis: Accurate determination ofribcage boundary in chest radiographs”, Xin-Wei Xu and Kunio Doi,Medical Physics, Volume 22(5), May 1995, pp. 617-626)) or the like maybe employed.

<1-3-1-3-2. Second Boundary Line Extracting Process>

A second boundary line extracting process is a process of extracting thediaphragm boundary line LI through model-based extraction. In otherwords, candidate positions of the diaphragm are roughly extractedthrough template matching that is one of model-based techniques (roughextraction), and the extracted candidate areas are analyzed in detail(fine extraction), whereby the diaphragm boundary line is extracted withhigh accuracy. For the rough extraction at this time, by using medicalknowledge relating to the motion of the diaphragm, templates can beweighted according to the amount of motion of the diaphragm, and theaccuracy of the rough extraction can be improved, whereby the extractionaccuracy of the diaphragm boundary line LI can be improved. As thisextraction method, for example, “Patent Application OPI No. 2012-138364(Application Date: Jun. 20, 2012)” that is an application filed by thepresent applicant may be employed.

<1-3-1-3-3. Third Boundary Extracting Process>

A third boundary line extracting process is a process of extracting thediaphragm boundary line LI through extraction according to a profileanalysis. FIG. 9 is a diagram that illustrates the profile analysis.FIG. 9A is a diagram that illustrates a profile area R2 of the selectiontarget frame image TI, and FIG. 9B is a graph of which the vertical axisrepresents a gray value for vertical coordinates the profile area R2(see FIG. 9A) of the selection target frame image TI of the horizontalaxis. As illustrated in FIG. 9, the profile R2 is generated in thevertical direction, and points at which the peak of the gray valuechanges in the generated profile R2 can be extracted as a boundary lineof the diaphragm.

<1-3-1-3-4. Fourth Boundary Extracting Process>

A fourth boundary line extracting process is a process of extracting thediaphragm boundary line LI through extraction according to a user'sdesignation. More specifically, the user designation may be made bysimply allowing a user to draw a line of the extraction target of thediaphragm boundary line LI or may be used as a method for correcting thediaphragm boundary line LI extracted by the first to third boundary lineextracting processes. In addition, in the former case where the line issimply designated by the user, it is preferable that only one of theselection target frame images TI that are targets is designated by theuser, and the remaining frame images are traced by employing acorresponding point search method in the time direction or the like.

<1-3-1-4. Displacement Correcting Unit 140>

FIG. 10 is a schematic view that illustrates a displacement amountbetween diaphragm boundary lines LI in the selection target frame imageTI extracted by the boundary line extracting unit 130. FIGS. 10A and 10Billustrate diaphragm boundary lines L1 and L2 in frame images T1 and T2extracted by the boundary line extracting unit 130, and FIG. 10Cdisplays the diaphragm boundary lines L1 and L2 in an overlappingmanner.

FIG. 11 is a schematic view that illustrates two examples of thedisplacement amount between the diaphragm boundary lines LI in theselection target frame image TI extracted by the boundary lineextracting unit 130. FIG. 11A illustrates a deformation according to avertical motion as an example, and FIG. 11B illustrates deformationsaccording to a parallel motion and rotation as an example.

In other words, in changes of the shape of the diaphragm boundary lineLI between the selection target frame images TI extracted by theboundary line extracting unit 130 as illustrated in FIG. 11, two kindsof changes including (i) a motion in which the shape of the diaphragmboundary line LI changes, in other words, a “shape deformation” and (ii)a motion accompanying a respiration motion of the diaphragm, in otherwords, a “deformation according to a vertical motion, a parallel motion,and rotation” as illustrated in FIGS. 11A and 11B are included. Of thedeformations, in order to acquire only the shape deformation of (i), itis necessary to remove the deformation (see FIG. 11) according to thevertical motion, the parallel motion, and the rotation of (ii) from thediaphragm boundary line LI.

Thus, the displacement correcting unit 140 executes the following twoprocesses. As a first process, a displacement amount calculating processis executed in which, by using pixels corresponding to a first number (aplurality of) diaphragm boundary lines LI, a displacement amount Dacquired by using the base boundary line BL as a displacement base iscalculated for one or more diaphragm boundary lines LI other than thebase boundary line BL among the predetermined first number of diaphragmboundary lines LI. Here, the displacement amount D represents aremoval-required component, and the removal-required component includesat least one component of deformation components according to a verticalmotion, a parallel motion, and rotation in the diaphragm area.

Next, as a second process, a correction process is executed in which,after the displacement amount calculating process, a second number (apredetermined number acquired by excluding the base boundary line BL),which is the first number or less, of the diaphragm boundary lines LIare corrected by using the displacement amount D. By executing such twoprocesses, the second number (predetermined number) ofdisplacement-corrected boundary lines LIc from which theremoval-required component has been removed are acquired (see FIG. 3).

In other words, the displacement correcting unit 140 receives the baseboundary line BL and the reference boundary line RL extracted by theboundary line extracting unit 130 and executes the displacement amountcalculating process and the correction process. Here, in thedisplacement amount calculating process, a displacement amount betweenpixels corresponding to the base boundary line BL and the referenceboundary line RL is calculated. Then, in the correction process, acorrection process is executed for the diaphragm boundary line LI thatis the correction target by using the displacement amount, whereby adisplacement-corrected boundary line LIc is acquired. Here, thediaphragm boundary line LI that is the correction target corresponds tothe reference boundary line RL.

FIG. 12 is a schematic view that illustrates a displacement-correctedboundary line LIc at the time of the exhalation phase PH2 (see FIG. 5).The displacement-corrected boundary lines L1 c to L5 c (LIc) illustratedin FIG. 12 correspond to boundary lines acquired by removing theremoval-required components from the diaphragm boundary lines L1 to L5(LI) (in other words, the diaphragm boundary line LI that is thecorrection target) illustrated in FIGS. 6 and 7 by using thedisplacement correcting unit 140. Here, in the example illustrated inFIG. 12, since the base boundary line BL is set as the diaphragmboundary line L1, the frame selecting unit 120 selects the base frameimage BF as the frame image T1 and selects the reference frame images RF(frame images that are correction targets) as the frame images T1 to T5.In other words, the displacement correcting unit 140 executes thecorrection process with the base boundary line BL set as the diaphragmboundary line L1 and the reference boundary line RL being set as thediaphragm boundary lines L1 to L5, thereby acquiringdisplacement-corrected boundary lines L1 c to L5 c (LIc).

Hereinafter, after four methods will be specifically described for thedisplacement amount calculating process, two methods will bespecifically described for the correction process. This embodiment isnot limited to those methods, but any other method may be used as longas the displacement amount calculating process and the correctionprocess can be appropriately executed.

<1-3-1-4-1. First Displacement Amount Calculating Process>

A first displacement amount calculating process is effective only in acase where the deformation of (ii) described above is due to only thevertical motion. In other words, the process is executed under thepremise that the displacement amount between the diaphragm boundarylines LI is only in the vertical direction.

FIG. 13 is a schematic view that illustrates a first displacement amountcalculating process. As illustrated in FIG. 13, a case will beconsidered in which the base boundary line BL is set as the diaphragmboundary line L1, the reference boundary line RL is set as the diaphragmboundary line L2, and pixels of the diaphragm boundary line L2 thatcorrespond to pixels P11, P12, P13, and P14 of the diaphragm boundaryline L1 are pixels P21, P22, P23, and P24. Here, for the convenience ofdescription, while only four corresponding pixels are representativelyillustrated for each of the base boundary line BL and the referenceboundary line RL, actually, corresponding pixels of the base boundaryline BL and the reference boundary line RL are appropriately designated.

At this time, a displacement amount of the diaphragm boundary line L2with respect to the diaphragm boundary line L1, in other words, adisplacement amount of a corresponding pixel is only in the Y-axisdirection. In other words, a “displacement amount d1 between the pixelsP11 and P21”, a “displacement amount d2 between the pixels P12 and P22”,a “displacement amount d3 between the pixels P13 and P23”, and a“displacement amount d4 between the pixels P14 and P24” calculated inthe first displacement amount calculating process are only Y-componentvalues.

For example, instead of using the values of the displacement amounts d1to d4 as the displacement amount D12 between the base boundary line BLand the reference boundary line RL, an average value, a minimal value,or a maximum value of the displacement amounts d1 to d4 may be used asthe displacement amount D12 between the base boundary line BL and thereference boundary line RL. This displacement amount D12 is a verticalmotion deformation component between the base boundary line BL and thereference boundary line RL.

<1-3-1-4-2. Second Displacement Amount Calculating Process>

A second displacement amount calculating process is effective in a casewhere the deformation of (ii) described above is due to not only thevertical motion but also a parallel motion or rotation. In other words,the process is executed under the premise that the displacement amountbetween the diaphragm boundary lines LI includes all the verticalmotion, the parallel motion, and the rotation.

FIG. 14 is a schematic view that illustrates a second displacementamount calculating process. Reference signs used in FIG. 14 are commonto those used in FIG. 13. A difference of the case illustrated in FIG.14 from that illustrated in FIG. 13 is that a method of designating acorresponding pixel is different. In other words, in the seconddisplacement amount calculating process, the displacement amount D12 iscalculated such that endpoints match each other. More specifically,rotation and parallel movement are made such that the end points of thediaphragm boundary lines L1 and L2, in other words, the pixels P10 andP20 and the pixels P14 and P24 match each other, and displacementamounts d1 to d4 are calculated for coordinate points present betweenthe end points.

For example, instead of using values of parallel motion amounts orrotation angles that can be calculated based on the displacement amountsd1 to d4 illustrated in FIG. 14 as the displacement amount D12 betweenthe base boundary line BL and the reference boundary line RL, an averagevalue (a minimal value or a maximum value) of the parallel motionamounts or the rotation angles or an amount requested for a combinationthereof may be used as the displacement amount D12 between the baseboundary line BL and the reference boundary line RL. This displacementamount D12 is a deformation component according to the parallel motionand the rotation between the base boundary line BL and the referenceboundary line RL.

In addition, the displacement amounts d1 to d4 between pixelsillustrated in FIG. 14 correspond to distances between points that areclosest to each other. Here, when rotation or a parallel motion is made,an affine transformation or the like may be employed. In addition, byemploying a least square method, distances between points that areclosest to each other may be acquired.

<1-3-1-4-3. Third Displacement Amount Calculating Process>

A third displacement amount calculating process is effective in a casewhere the deformation of (ii) described above is due to not only thevertical motion but also a parallel motion or rotation. In other words,the process is executed under the premise that the displacement amountbetween the diaphragm boundary lines LI includes all the verticalmotion, the parallel motion, and the rotation.

In the third displacement amount calculating process, a displacementamount D is calculated through shape fitting. As a specific method offitting calculation, for example, an iterative closest point (ICP)algorithm or the like may be used. When the ICP algorithm is used, byparallel moving or rotating one boundary line of the base boundary lineBL and the reference boundary line RL, convergence calculation can beexecuted such that a distance between corresponding pixels is minimal.Here, while the ICP algorithm has been described as an example, anyother method executing fitting such that a distance betweencorresponding pixels is minimal through convergence calculation may beused. As an advantage of executing fitting through convergencecalculation such as the ICP algorithm, more detailed shape matching canbe made, and accordingly, an accurate displacement amount D can becalculated.

In this way, a displacement amount D between the base boundary line BLand the reference boundary line RL is acquired such that a distancebetween corresponding pixels is minimal through the fitting process.This displacement amount D is a deformation component according to theparallel motion and the rotation between the base boundary line BL andthe reference boundary line RL.

<1-3-1-4-4. Fourth Displacement Amount Calculating Process>

A fourth displacement amount calculating process is effective in a casewhere the deformation of (ii) described above is due to not only thevertical motion but also a parallel motion or rotation. In other words,the process is executed under the premise that the displacement amountbetween the diaphragm boundary lines LI includes all the verticalmotion, the parallel motion, and the rotation.

In the fourth displacement amount calculating process, eachcorresponding pixel of the reference boundary line RL for the baseboundary line BL is tracked, and a “tracking result” is calculated as adisplacement amount D. Asa specific tracking calculation method, amethod as described below may be employed.

For example, a POC (phase restricting correlation) may be employed. Inthe POC algorithm, a correlation (similarity) between a registrationimage (base boundary line BL) that becomes a background and an inputimage (reference boundary line RL) to be checked is calculated. Morespecifically, when an image formed as a digital signal is mathematicallyprocessed through a Fourier transform, the image is decomposed into anamplitude (gray data) and a phase (contour data of the image). Out ofthe two kinds of information, the amplitude information in which shapeinformation is not included is not used, and image processing of thecorrelation can be instantly executed by using only the phaseinformation. Accordingly, as an effect, a high-frequency component canbe cut off, and erroneous detection of corresponding points due to theinfluence of a blood flow can be prevented.

In addition, a rotation invariant phase only correlation (RIPOC) mayalso be employed. In the RIPOC algorithm, a rotation amount estimatingprocess is considered in which, with respect to an image (base boundaryline BL) that becomes a base, the degree of rotation of another image(reference boundary line RL) that is a comparison target is detected.More specifically, in a case where an image is rotated, naturally, thefrequency component of the image changes. While a change of the phasecomponent is complicated, the amplitude component is rotated inaccordance with the rotation of the image, and the change thereof doesnot depend on the position of the rotation center. Thus, in the RIPOC,focusing on the characteristic of this amplitude component, a polarcoordinate image of which the X direction is an angle theta and the Ydirection is the radius r is generated by converting the amplitudecomponent into polar coordinates. Then, by performing matching betweenpolar-coordinate images, the deviation in the X direction corresponds tothe angle deviation. Accordingly, the rotation amount can be estimatedbased on a result of the matching, and the original image is correctedby using the estimated rotation amount, and thereafter, the position canbe estimated. For example, a method as disclosed in Japanese Patent No.3574301 may be employed.

In this way, by employing the POC or the RIPOC, a tracking resultacquired by a corresponding point search for each pixel of the baseboundary line BL can be set as a displacement amount D between the baseboundary line BL and the reference boundary line RL. This displacementamount D is a deformation component according to the vertical motion,the parallel motion, and the rotation between the base boundary line BLand the reference boundary line RL.

In addition, while the first to fourth displacement amount calculatingprocesses described above have been described for a case where thedisplacement amount D (D12) between the base boundary line BL and thereference boundary line RL is acquired as an example, in a case where aplurality of reference boundary lines RL are present, it is apparentthat a displacement amount D between the reference boundary lines RL andRL can be acquired.

In the first to fourth displacement amount calculating processesdescribed above, the displacement amount D is acquired so as to approachand match the base boundary line BL not only in a case where thedisplacement amount D between the base boundary line BL and thereference boundary line RL is acquired, but also in a case where thedisplacement amount D between the reference boundary lines RL and RL isacquired. In addition, in the first and second displacement amountcalculating processes, the displacement amount D may be acquired bysubtracting pixels corresponding to each other from each other. Thus,all the first to fourth displacement amount calculating processesdescribed above are processes for acquiring the displacement amount Dwith the base boundary line BL being set as a displacement base.

<1-3-1-4-5. First Correction Process>

Subsequently, the correction process executed after the displacementamount calculating process will be described.

A first correction process is a process in which the diaphragm boundaryline LI is corrected by using a displacement amount D12 calculated usingthe base boundary line BL as the diaphragm boundary line L1 and usingthe reference boundary line RL as the diaphragm boundary line L2.

FIG. 15 is a schematic view that illustrates the first correctionprocess. In FIG. 15, for the convenience of description, thedisplacement amount D of the case of only a vertical motion is assumed,and, as corresponding pixels of the displacement amount D12, pixels P1and P2 are representatively illustrated.

As illustrated in FIG. 15, first, in a case where the base boundary lineBL is the diaphragm boundary line L1, and the reference boundary line RLthat is the diaphragm boundary line LI, which is a correction target, isthe diaphragm boundary line L2, the displacement amount D12 between thediaphragm boundary lines L1 and L2 is calculated by the firstdisplacement calculating process described above. Then, the diaphragmboundary lines L2 and L3 are corrected by using this displacement amountD12. More specifically, in a case where a displacement-correctedboundary line L2 c is acquired by correcting the diaphragm boundary lineL2, in other words, the pixel P2 on the diaphragm boundary line L2becomes a pixel P2 c on the displacement-corrected boundary line L2 cafter the first correction process.

In addition, also in a case where the diaphragm boundary line L3 iscorrected by using the displacement amount D12 (the diaphragm boundaryline LI that is the correction target is the diaphragm boundary lineL3), the pixel P3 on the diaphragm boundary line L3 becomes a pixel P3 con the displacement-corrected boundary line L3 c after the firstcorrection process.

By executing the first correction process described above,displacement-corrected boundary lines L2 c to L5 c (LIc) illustrated inFIG. 12 described above are acquired.

In the example described above, while the first correction process usingthe displacement amount D12 between the diaphragm boundary lines L1 andL2 acquired by the first displacement amount calculating process hasbeen illustrated, it is apparent that the first correction process maybe executed using the displacement amount D12 between the diaphragmboundary lines L1 and L2 acquired by the second to fourth displacementamount calculating processes.

<1-3-1-4-6. Second Correction Process>

A second correction process is a process in which the diaphragm boundaryline LI is corrected by using a displacement amount D1I calculated usingthe base boundary line BL as the diaphragm boundary line L1 and usingthe reference boundary line RL as the diaphragm boundary line LI (here,an argument I is an integer of two or more).

FIG. 16 is a schematic view that illustrates the second correctionprocess. In FIG. 16, for the convenience of description, thedisplacement amount D of the case of only a vertical motion is assumed,and, as corresponding pixels of the displacement amount D12, pixels P1and P2 are representatively illustrated, and, as corresponding pixels ofa displacement amount D13, the pixel P1 and the pixel P3 arerepresentatively illustrated.

As illustrated in FIG. 16, first, in a case where the base boundary lineBL is the diaphragm boundary line L1, and the reference boundary lineRL, which is a correction target, is the diaphragm boundary line L2, thedisplacement amount D12 between the diaphragm boundary lines L1 and L2is calculated by the first displacement calculating process describedabove. Then, the diaphragm boundary line L2 is corrected by using thisdisplacement amount D12. More specifically, similar to the descriptionpresented above, the pixel P2 on the diaphragm boundary line L2 becomesa pixel P2 c on the displacement-corrected boundary line L2 c after thesecond correction process.

In addition, when the diaphragm boundary line L3 is corrected by usingthe displacement amount D13 between the base boundary line BL and thediaphragm boundary line L3 (in a case where the diaphragm boundary lineLI that is the correction target is the diaphragm boundary line L3), thepixel P3 on the diaphragm boundary line L3 becomes a pixel P3 c on thedisplacement-corrected boundary line L3 c after the second correctionprocess.

FIG. 17 is a schematic view that illustrates before and after the secondcorrection process. FIG. 17A illustrates the diaphragm boundary lines L2and L3 before the process (the diaphragm boundary line LI is thecorrection target), and FIG. 17B illustrates the base boundary line BL(diaphragm boundary line L1) and displacement-corrected boundary linesL2 c and L3 c after the correction process.

By executing the second correction process described as above, asillustrated in FIG. 17B, a correction result is acquired in which thedisplacement-corrected boundary lines L2 c and L3 c approach and matchthe base boundary line BL (diaphragm boundary line L1) that is thedisplacement base.

A correction process including the first and second correction processesdescribed above is as follows. In the correction process, a displacementamount calculating process in which, by using a pixel corresponding to atarget area boundary line, for one or more of target area boundary linesother than the base boundary line BL (L1) among a plurality of thetarget area boundary lines (diaphragm boundary lines L1 to L3), adisplacement amount ΔD (D12 or D12 and D13) is calculated using the baseboundary line BL as a displacement base is executed, the displacementamount ΔD is set as a removal-required component, and two (apredetermined number of) target area boundary lines L2 and L3 other thanthe base boundary line BL are corrected by using the displacement amountΔD after the displacement amount calculating process.

In the first correction process, the displacement amount ΔD used for thecorrection process is the displacement amount D12 between pixelscorresponding to the base boundary line and one (diaphragm curve L2)among target area boundary lines. A process in which target areaboundary lines (diaphragm curve L2) other than the one target areaboundary line are corrected by using this displacement amount D12 is thefirst correction process.

Meanwhile, in the second correction process, the displacement amounts ΔDused for the correction process are displacement amounts D12 and D13between corresponding pixels of the base boundary line BL (L1) and oneor more (diaphragm curves L2 and L3) of the target area boundary lines.A process in which two (a predetermined number of) target area boundarylines (diaphragm curves L2 and L3) other than the base boundary line BLare corrected by using these displacement amounts D12 and D13 is thesecond correction process.

<1-3-1-5. Image Generating Unit 150 and Display Unit 34>

The image generating unit 150 generates displacement-corrected boundaryline information LG for display based on a second number, which is thefirst number or less, of the displacement-corrected boundary lines LIc(see FIG. 3). As the displacement-corrected boundary line informationLG, the second number of separate images that are separated for each ofthe second number of the displacement-corrected boundary lines LIc ormay generate one still image such that the second number of thedisplacement-corrected boundary lines LIc are displayed in anoverlapping manner.

In addition, it is preferable that information indicating the baseboundary line BL is included in the displacement-corrected boundary lineinformation LG.

Then, the display unit 34 displays the displacement-corrected boundaryline information LG generated by the image generating unit 150 (see FIG.3). In other words, in a case where separate images are generated, thedisplay unit 34 sequentially displays the second number of the separateimages as the displacement-corrected boundary line information LG. Onthe other hand, in a case where a still image is generated, the displayunit 34 displays one still image as the displacement-corrected boundaryline information LG.

FIGS. 18 and 19 are schematic views that illustrate thedisplacement-corrected boundary line information LG as an example. FIG.18 illustrates the displacement-corrected boundary line information LGfor a healthy person, and FIG. 19 is a diagram that illustrates thedisplacement-corrected boundary line information LG for an unhealthyperson. In addition, FIGS. 18A and 19A illustrate a case where the imagegenerating unit 150 generates one still image (thedisplacement-corrected boundary line information LG for display) bychanging the color for each boundary line such that thedisplacement-corrected boundary lines LIc are displayed in anoverlapping manner based on the displacement-corrected boundary lines L2c to L5 c (LIc) of which the second number of “4” and displays thegenerated still image on the display unit 34. In addition, similar tothe example illustrated in FIGS. 18A and 19A, information indicating thebase boundary line BL may be included inside the displacement-correctedboundary line information LG so as to simultaneously display thediaphragm boundary line L1 that is the base boundary line BL in anoverlapping manner.

On the other hand, FIGS. 18B and 19B illustrate a case where the imagegenerating unit 150 generates an average displacement-corrected boundaryline LAc calculated based on the displacement-corrected boundary linesL2 c to L5 c as the displacement-corrected boundary line information LGfor display and displays the generated average displacement-correctedboundary line on the display unit 34.

As illustrated in FIGS. 18A and 19A, in the displacement-correctedboundary lines L2 c to L5 c, the deformation due to the vertical motion,the parallel motion, and the rotation of (ii) described above isremoved, and only the shape deformation of (i) described above isrepresented. In other words, like the diaphragm boundary lines L1 to L5illustrated in FIG. 7, not the whole change in the pulmonary field but apartial abnormality of the shape can be represented.

In addition, in FIG. 18A, the shape of the left side L and the rightside R (in other words, the shape of the left pulmonary field and theright pulmonary field) is horizontally symmetrical. However, in FIG.19A, the displacement-corrected boundary lines L2 c to L5 c on the rightside R of the diaphragm are at the almost same positions, and the shapeof the left side L and the right side R of the diaphragm isasymmetrical. By displaying the displacement-corrected boundary line LIcin this way, in the case of a healthy examinee M, as illustrated in FIG.18A, a shape that is horizontally symmetrical is displayed in a regularradial pattern. However, in the case of an examinee M who has sufferedfrom illness, a deformed shape suggesting an abnormality in part isdisplayed. For this reason, instead of making a diagnosis based on theuser's subjectivity, an objective diagnosis can be made.

Furthermore, as illustrated in FIGS. 18B and 19B, one line that is basedon an average value can be output and displayed. Thus, as the userexamines the average displacement-corrected boundary line LAc, the shapeof the displacement-corrected boundary lines L2 c to L5 c can bediagnosed comprehensively.

In the example of the displacement-corrected boundary line informationLG described above, while one still image has been described to begenerated such that the displacement-corrected boundary lines LIc aredisplayed, separate images separated for each displacement-correctedboundary line LIc may be generated. In other words, in the case of theexamples illustrated in FIGS. 18 and 19, by generating four separateimages separated for each of the displacement-corrected boundary linesL2 c to L5 c, the separate images may be sequentially displayed as adynamic image. In addition, only designated displacement-correctedboundary lines LIc may be displayed.

In addition, the image generating unit 150, based on thedisplacement-corrected boundary line LIc may calculate a degree ofvariations (for example, dispersion/a total distance difference, anaverage, a maximal distance difference, or the like) among thedisplacement-corrected boundary lines LIc and display the degree ofvariations as the displacement-corrected boundary line information LG onthe display unit 34. For example, the displacement-corrected boundaryline information LG may be displayed as two displacement-correctedboundary lines LIc having highest degrees of variations, or onlydisplacement-corrected boundary lines LIc of which the degrees ofvariations are a threshold or more may be displayed.

<1-4. Basic Operation of Image Processing Device 3>

FIG. 20 is a flowchart that illustrates the basic operation realized inthe image processing device 3 according to this embodiment. Theindividual function of each unit has already been described (see FIG.3), here, only the whole process will be described.

As illustrated in FIG. 20, first, in Step S1, the dynamic imageacquiring unit 110 of the control unit 31 acquires a dynamic image (aplurality of frame images MI) photographed by the reading control device14 of the photographing apparatus 1 through the photographing controldevice 2.

In Step S2, the frame selecting unit 120 executes the frame selectingprocess including the process of selecting the base frame image BF usedfor extracting the base boundary line BL and the reference frame imagesRF used for extracting a predetermined number of diaphragm boundarylines LI other than the base boundary line BL for the selection targetframe images TI (see FIG. 3). In the example illustrated in FIG. 7, theframe image T1 is selected as the base frame image BF, and frame imagesT2 to T5 are selected as the reference frame images RF.

In Step S3, in the boundary line extracting unit 130 executes theboundary line extracting process for acquiring the diaphragm boundarylines LI (in other words, the base boundary line BL and the referenceboundary lines RL) by extracting boundary lines of the diaphragm for thebase frame image BF and the reference frame images RF (the diaphragmboundary line LI that is the correction target) selected in Step S2 (seeFIGS. 6 and 7). In the example illustrated in FIG. 7, the diaphragmboundary line L1 is extracted as the base boundary line BL and thediaphragm boundary lines L2 to L5 are extracted as the referenceboundary lines RL that are the diaphragm boundary lines LI of thecorrection targets.

In Step S4, the displacement correcting unit 140, by using pixelscorresponding to the base boundary line BL and the reference boundarylines RL extracted in Step S3, executes the displacement amountcalculating process (any one of the first to fourth displacement amountcalculating processes) for calculating a displacement amount D of thereference boundary line RL by using the base boundary line BL as adisplacement base and then executes the correction process forcorrecting the diaphragm boundary line LI that is the correction targetby using the displacement amount D. Accordingly, thedisplacement-corrected boundary lines LIc from which theremoval-required component (the deformation component of a verticalmotion, a parallel motion, rotation, or the like) is removed areacquired (FIGS. 12 and 17B)).

In Step S5, in a case where the process is further executed by thedisplacement correcting unit 140, when the reference boundary line RL(the diaphragm boundary line LI that is the correction target) ischanged, the displacement correcting unit 140 instructs the frameselecting unit 120 to change the reference frame images RF (the frameimages that are the correction targets), and the process of Steps S2 toS4 is repeated again. On the other hand, in a case where the process isended by the displacement correcting unit 140, the process proceeds toStep S6.

In other words, in Step S2, as illustrated in FIG. 15, in a case wherethe reference frame images RF (frame images that are correction targets)are frame images T2 and T3, and the frame images T2 and T3 are selectedtogether (in other words, the reference boundary lines RL are thediaphragm boundary lines L2 and L3), the process proceeds to Step S6.However, in a case where the frame images T2 and T3 are individuallyselected instead of being selected together as the reference frameimages RF (in other words, in a case where the diaphragm boundary linesL2 and L3 as the reference boundary lines RL are not processedtogether), when the reference frame image RF is changed, the processproceeds to Step S2, and the process of Steps S2 to S4 is executed.

In Step S6, the image generating unit 150 generates thedisplacement-corrected boundary line information LG based on thedisplacement-corrected boundary lines LIc acquired in Step 4 (see FIGS.18 and 19).

In the displacement-corrected boundary line information LG, at leastinformation indicating the displacement-corrected boundary lines LIc isincluded, and it is preferable to also include information indicatingthe base boundary line BL therein.

Finally, in Step S7, the image generating unit 150 outputs thedisplacement-corrected boundary line information LG generated in Step S6to the display unit 34 or the storage unit 32 (see FIG. 3), and the flowof this operation ends.

As above, in the image processing device 3 according to the firstembodiment, the displacement amount calculating process is executed inwhich, by using pixels corresponding to the first number of (a pluralityof) diaphragm boundary lines LI (a plurality of target area boundarylines), a displacement amount D acquired by using the base boundary lineBL as a displacement base is calculated for one or more diaphragmboundary lines LI other than the base boundary line BL among the firstnumber of diaphragm boundary lines LI (a plurality of reference boundarylines RL). Here, the displacement amount D represents a removal-requiredcomponent. Then, by executing the correction process for correcting apredetermined number (the second number) of diaphragm boundary lines LIother than the base boundary line BL by using the displacement amount Dafter the displacement amount calculating process, the second number(predetermined number) of displacement-corrected boundary lines LIc inwhich the removal-required component is removed are acquired, and thedisplacement-corrected boundary line information LG for display that isbased on the second number (predetermined number) of thedisplacement-corrected boundary lines LIc is displayed on the displayunit 34. In other words, by displaying the displacement-correctedboundary lines LIc acquired by removing the deformation corresponding tothe displacement amount D from the diaphragm boundary lines LI, the userperceives a change in the shape of the diaphragm boundary line LI,whereby the motion of the shape of the diaphragm can be perceived. Inaddition, since the shape of the diaphragm boundary line LI can beperceived, a partial abnormality of the shape can be found, and thepartial abnormality such as adhesion can be easily diagnosed.Furthermore, since the diagnosis content desired by the user iscollected in the displacement-corrected boundary line information LG, aminimum requisite diagnosis time is necessary, whereby the efficiency ofthe diagnosis is improved. For this reason, a dynamic-state diagnosiscan be executed appropriately and efficiently.

In addition, the removal-required component includes at least onecomponent among deformation components according to a vertical motion, aparallel motion, and rotation in the target area, and accordingly, thedisplacement-corrected boundary lines LIc acquired by removing thedeformations due to the vertical motion, the parallel motion, and therotation from the diaphragm boundary lines LI can be displayed.

Furthermore, the frame selecting unit 120 is further included whichexecutes, for selection target frame images TI including at least aplurality of frame images MI, the frame selecting process including theprocess of selecting the base frame image BF used for extracting thebase boundary lines BL and the reference frame images RF used forextracting the diaphragm boundary lines LI other than the base boundaryline BL, and the displacement calculating process includes the processof calculating a displacement amount D between corresponding pixels ofthe diaphragm boundary lines RL of the reference frame image RF by usingthe diaphragm boundary line LI of the base frame image BF as the baseboundary line BL. Accordingly, frame images corresponding to user'sdiagnosis purpose can be selected, and the displacement-correctedboundary line information LG corresponding to the diagnosis purpose canbe displayed. In addition, by selecting only necessary frame images,compared to the case where the displacement-corrected boundary lines LIcare acquired for all the frame images MI included in the dynamic image,a calculation time required for the boundary line extracting process,the displacement amount calculating process, and the correction processcan be minimized.

In addition, the second number (predetermined number) of separate imagesseparated for each of the second number (predetermined number) ofdisplacement-corrected boundary lines LIc are generated, and the secondnumber (predetermined number) of separate images are sequentiallydisplayed as the displacement-corrected boundary line information LG.Accordingly, a change of the shape of the diaphragm boundary line LI canbe perceived from the dynamic image.

Furthermore, one still image is generated, and the one still image isdisplayed as the displacement-corrected boundary line information LGsuch that the second number (predetermined number) ofdisplacement-corrected boundary lines LIc are displayed in anoverlapping manner. For example, in a case where the shape of the secondnumber (predetermined number) of the displacement-corrected boundarylines LIc set as diagnosis targets is used as the displacement-correctedboundary line information LG, the displacement-corrected boundary linesLIc can be displayed to be identifiable in an overlapping manner.Accordingly, a change of the shape of the diaphragm boundary line LI canbe perceived on one still image.

In addition, since the target area is the diaphragm area, diseases suchas pulmonary emphysema and diaphragmatic eventration can beappropriately diagnosed through a dynamic-state diagnosis. In such adisease, in the case of a minor symptom, while the abnormality may notbe noticed, by making a diagnosis using the displacement-correctedboundary line information LG, the diagnosis does not depend on theuser's subjectivity, whereby an erroneous diagnosis can be prevented.

2. Second Embodiment

In an image processing device according to a second embodiment of thepresent invention, a displacement correcting unit is different from thatof the image processing device 3 according to the first embodiment inpoints to be described below. In addition, the remaining configurationis similar to that of the image processing device 3.

<2-1. Displacement Correcting Unit>

Hereinafter, a correction process executed by the displacementcorrecting unit according to the second embodiment will be referred toas a third correction process. A displacement amount D used for thethird correction process is a displacement amount from a diaphragmboundary line LI that is nearest in time to a diaphragm boundary line LIthat is a correction target.

FIG. 21 is a schematic view that illustrates the third correctionprocess. In FIG. 21, for diaphragm boundary lines L1 to L4, for theconvenience of description, a displacement amount D of the case of onlya vertical motion is assumed, and, as corresponding pixels of adisplacement amount D12, pixels P1 and P2 are representativelyillustrated, as corresponding pixels of a displacement amount D23, thepixel P2 and a pixel P3 are representatively illustrated, and, ascorresponding pixels of a displacement amount D34, the pixel P3 and apixel P4 are representatively illustrated.

As illustrated in FIG. 21, first, in a case where a base boundary lineBL is a diaphragm boundary line L1, and a diaphragm boundary line LI,which is a correction target, in other words, a reference boundary lineRL is a diaphragm boundary line L2, similar to the second correctionprocess, the diaphragm boundary line L2 is corrected by using thedisplacement amount D12 between the diaphragm boundary lines L1 and L2.More specifically, similar to the description presented above, the pixelP2 on the diaphragm boundary line L2 becomes a pixel P2 c on thedisplacement-corrected boundary line L2 c after the third correctionprocess.

Next, when a diaphragm boundary line LI that is a correction target isthe diaphragm boundary line L3, a comparison target for which thedisplacement amount is acquired is changed to the diaphragm boundaryline L2, and the diaphragm boundary line L3 is corrected using thedisplacement amount D23. In such a case, the pixel P3 on the diaphragmboundary line L3 becomes a pixel P3 c on the displacement-correctedboundary line L3 c after the third correction process.

Furthermore, when a diaphragm boundary line LI that is a correctiontarget is the diaphragm boundary line L4, a comparison target for whichthe displacement amount is acquired is changed to the diaphragm boundaryline L3, and the diaphragm boundary line L4 is corrected using thedisplacement amount D34 between the diaphragm boundary lines L3 and L4.At this time, the pixel P4 on the diaphragm boundary line L4 becomes apixel P4 c on the displacement-corrected boundary line L4 c after thethird correction process.

As above, in the image processing device according to the secondembodiment, the displacement amount D used for the correction process isa displacement amount D from the diaphragm boundary line LI that isnearest in time from the diaphragm boundary line LI that is a correctiontarget.

Thus, by executing the third correction process constantly using thedisplacement amount D between the diaphragm boundary lines LI ofselection target frame images TI that are nearest to each other,displacement-corrected boundary lines L2 c to L4 c having highcorrection accuracy can be acquired. As a result, thedisplacement-corrected boundary line information LG that is suitable forthe diagnosis purpose can be displayed, and accordingly, thedynamic-state diagnosis can be executed more appropriately andefficiently.

3. Third Embodiment

In an image processing device according to a third embodiment of thepresent invention, a displacement correcting unit is different from thatof the image processing device 3 according to the first embodiment inpoints to be described below. In addition, the remaining configurationis similar to that of the image processing device 3.

<3-1. Displacement Correcting Unit>

Hereinafter, a correction process executed by the displacementcorrecting unit according to the third embodiment will be referred to asa fourth correction process. In a case where at least one diaphragmboundary line LI is present between a base boundary line BL to adiaphragm boundary line LI that is a correction target, a displacementamount D used for the fourth correction process is a displacement amountD between the base boundary line BL to the diaphragm boundary line LI,which is the correction target, that is acquired as a sum ofdisplacement amounts D between two boundary lines LI that are adjacentin time.

FIG. 22 is a schematic view that illustrates the fourth correctionprocess. In FIG. 22, for diaphragm boundary lines L1 to L3, for theconvenience of description, as corresponding pixels of a displacementamount D12, pixels P1 and P2 are representatively illustrated, ascorresponding pixels of a displacement amount D23, the pixel P2 and apixel P3 are representatively illustrated, and, as corresponding pixelsof a displacement amount D13, the pixel P1 and the pixel P3 arerepresentatively illustrated.

As illustrated in FIG. 22, a case will be described in which thediaphragm boundary line LI that is a correction target is the diaphragmboundary line L3, and a correction is made using the displacement amountD13. In the fourth correction process, in a displacement amountcalculating process, the displacement amount D13 is calculated withbeing subdivided into the displacement amount D12 and the displacementamount D23, and a displacement amount (D12+D23) acquired by adding boththe displacement amounts D is used as a displacement amount between thediaphragm boundary lines L1 and L3. For this reason, first, by thedisplacement amount calculating process, the displacement amount D12between the diaphragm boundary line L1 and the diaphragm boundary lineL2 is calculated, and the displacement amount D23 between the diaphragmboundary line L2 and the diaphragm boundary line L3 is calculated. Inthis way, in the displacement amount calculating process, (D12+D23) iscalculated.

In other words, when the displacement amount D13 is calculated, comparedto the displacement amount D13 that is directly acquired between thediaphragm boundary line L1 and the diaphragm boundary line L3, similarto the first and second embodiments, in the displacement amount(D12+D23) calculated in this embodiment, the displacement amountacquired through the diaphragm boundary line L2 is included, andaccordingly, the route in which the diaphragm boundary lines L1 to L3are displaced is reflected, and the accuracy is high.

In this way, in the fourth correction process, the diaphragm boundaryline L3 is corrected using the displacement amount (D12+D23) calculatedwith high accuracy, and the displacement-corrected boundary line L3 c iscalculated more accurately as well. In other words, the pixel P3 on thediaphragm boundary line L3 becomes a pixel P3 c on thedisplacement-corrected boundary line L3 c after the fourth correctionprocess.

<3-2. Basic Operation>

In the basic operation of the image processing device according to thethird embodiment, Steps S2 to S4 illustrated in FIG. 20 are differentfrom those of the first embodiment. In other words, in Step S2 of thefirst embodiment, while the frame selecting unit 120 executes the frameselecting process between the base frame image BF and the referenceframe image RF, in Step S2 of the third embodiment, in addition thereto,a frame image (hereinafter, referred to as an “intermediate frameimage”) present between the photographing timing of the base frame imageBF and the photographing timing of the reference frame image RF is alsoselected, which is different from the first embodiment. In other words,in the example illustrated in FIG. 22 in which the displacement amountD13 is calculated, a frame image T1 is selected as the base frame imageBF, and a frame image T3 is selected as the reference frame image RF,and a frame image T2 is also selected as the intermediate frame imagepresent between such photographing timings.

Then, in Step S3 of the third embodiment, the boundary line extractingprocess is executed also for the intermediate frame image in addition tothe base frame image BF and the reference frame image RF.

In addition, in Step S4 of the third embodiment, a displacement amountcalculating process is executed in which a displacement amount D of thereference boundary line RL with respect to the base boundary line BL iscalculated by using the diaphragm boundary line of the intermediateframe image, and then a correction process is executed in which thediaphragm boundary line LI that is a correction target is corrected byusing the displacement amount D (see FIG. 22).

As above, in the image processing device according to the thirdembodiment, the displacement amount D used for the correction process isa displacement amount D between the base boundary line BL to thediaphragm boundary line LI, which is a correction target, that isacquired as a sum of displacement amounts D between two boundary linesLI adjacent in time. In this way, in the displacement amount calculatingprocess, one displacement amount D between the base boundary line BL andthe diaphragm boundary line LI is subdivided and calculated, andaccordingly, compared to the displacement amount D calculated withoutsubdividing the base frame image BF to the reference frame image RF, thedisplacement amount can be calculated with high accuracy.

4. Fourth Embodiment

FIG. 23 is a diagram that illustrates the functional configuration of acontrol unit 31A used by an image processing device 3A according to afourth embodiment. This control unit 31A is used as a substitute for thecontrol unit 31 (see FIG. 3) of the image processing device 3 accordingto the first embodiment. A difference from that of the first embodimentis that the control unit 31A further includes a period classifying unit115. Thus, in accordance therewith, the frame selecting unit 120 ischanged to a frame selecting unit 120A. The remaining configuration issimilar to that of the image processing device 3.

<4-1. Period Classifying Unit 115>

The period classifying unit 115 detects a so-called respiration period(target area period) in which the diaphragm of the examinee M (body)synchronized with the photographing time, at which a plurality of frameimages MI are photographed by the dynamic image acquiring unit 110,periodically changes and classifies the plurality of frame images MI inunits of the respiration periods (units of target area periods). Then,the period classifying unit 115 outputs a plurality of frame images MI′after the classification in units of the respiration periods to theframe selecting unit 120 (see FIG. 23).

When the respiration period of the examinee M is detected, the periodclassifying unit 115 executes a respiratory information acquiringprocess for acquiring respiratory information and detects a respirationperiod PC, an inhalation phase PH1 and an exhalation phase PH2 based onthe respiratory information. Hereinafter, the respiratory informationacquiring process and the detection of the respiration period and thelike will be described.

<4-1-1. Respiratory Information Acquiring Process>

When a physical state value defined as a value representing a physicalstate of the diaphragm area that changes in time is referred to as arespiration vibration value, the respiratory information acquiringprocess is a process in which a respiration vibration value iscalculated based on a plurality of frame images MI configuring a dynamicimage acquired by the dynamic image acquiring unit 110, and therespiration vibration value is set as the respiratory information (seeFIG. 23).

As illustrated in FIG. 23, first, in the respiratory informationacquiring process, the respiration vibration value is calculated byusing the plurality of frame images MI acquired by the dynamic imageacquiring unit 110. More specifically, the respiration vibration valueis an index used for measuring a change in the size of the pulmonaryfield area according to respiration, and examples thereof include a“distance between characteristic points of the pulmonary field area (adistance from the pulmonary apex to the diaphragm or the like)”, an“area value (the size of the pulmonary field area) of the pulmonaryfield part”, the “absolute position of the diaphragm”, and a “pixeldensity value of the pulmonary field area”, and the like. Hereinafter, acase will be described as an example in which the respiration vibrationvalue is the “area value of the pulmonary field part” and the “distancebetween characteristic points of the pulmonary field area”.

In a case where the respiration vibration value is the “area value ofthe pulmonary field part”, the contour of the pulmonary field part isextracted, and the number of pixels surrounded by the contour may bedefined as the area of the pulmonary field part.

FIGS. 24A and 24B are schematic views that illustrate the extraction ofthe contour of the pulmonary field part as an example. In the extractionof the pulmonary field part, by using the extraction method describedwith reference to FIG. 8, each of the left and right sides may beextracted (see FIG. 24A), similar to the case illustrated in FIG. 8, ora contour (see FIG. 24B) including the area of the heart and the spinemay be extracted as the pulmonary field part.

In this way, in the respiratory information acquiring process, thecontour OL of the pulmonary field part is extracted by using a pluralityof acquired frame images MI, and the number of pixels inside theextracted area is detected as the area value of the pulmonary fieldpart, whereby the respiration vibration value is acquired (see FIGS. 24Aand 24B).

In a case where the respiration vibration value is the “distance betweencharacteristic points of the pulmonary field area”, a distance betweencharacteristic points of the pulmonary field area is calculated by usinga plurality of frame images MI. In other words, the pulmonary field partis extracted using a method similar to the method described above, twocharacteristic points are acquired from the extracted area, and adistance between the two points is acquired, whereby a respirationvibration value is detected. Then, a change in the distance (respirationvibration value) between the characteristic points is set as arespiration phase PH.

FIGS. 24C and 24D are exemplary diagrams that illustrate the positionsof characteristic points of a pulmonary field area in a case where thecontour OL of the pulmonary field part illustrated in FIG. 24A is used.In a case where a change in a length (pulmonary field length) from theupper end LT of the pulmonary area to the lower end LB thereof iscalculated, the example illustrated in FIG. 24C is an example in whichthe pulmonary apex is set as the upper end LT of the pulmonary area, andan intersection between a line drawn downward from the pulmonary apex inthe body axis direction and the diaphragm is extracted as the lower endLB. The example illustrated in FIG. 24D is an example in which thepulmonary apex is set as the upper end LT of the pulmonary area, and acostophrenic angle is extracted as the lower end LB.

In this way, in the respiratory information acquiring process, by usinga plurality of frame images MI that have been acquired, the contour OLof the pulmonary field area is extracted, and a distance betweencharacteristic points is detected based on the extracted area, wherebythe respiration vibration value is acquired (see FIGS. 24C and 24D).

Then, as illustrated in FIG. 5 described above, a respiration phase PHin which waveform data of the respiration vibration value acquired inthe respiratory information acquiring process is represented in a timeseries, in other words, a result of monitoring a calculated respirationvibration value such as the area value of the pulmonary field area orthe distance between characteristic points for every photographingtiming TM in the time direction can be acquired.

In this embodiment, while the respiratory information is acquired byusing photographed images, a measurement result acquired using anexternal device may be used. In such a case, information relating to therespiration period is input from the external device to the periodclassifying unit 115. As a method for the measurement using the externaldevice, for example, a device as disclosed in Japanese Patent No.3793102 may be used. In addition, a technique (for example, see“Unrestrained Respiration Monitoring for Sleeping Person Using FiberGrating Vision Sensor”, Aoki Hirooki and Nakajima Masato, Proceedings ofthe Society Conference of IEICE 2001, Proceedings of the SocietyConference of Information System, 320-321, 2001-08-29 or the like) formonitoring using a sensor configured by a laser beam and a CCD camera orthe like may be used. In other words, by using a method of detecting themotion of the chest of a subject M by using laser radiation, arespiration monitor belt, or the like or by detecting the air stream ofthe respiration using an air flow meter, the respiratory information canbe acquired, and such methods may be applied.

<4-1-2. Method of Detecting Respiration Period PC, Inhalation Phase PH1,and Exhalation Phase PH2>

Subsequently, a change in the respiration vibration value that isdetected in the respiratory information acquiring process is set as therespiration phase PH, and the respiration period PC, the inhalationphase PH1, and the exhalation phase PH2 are detected. More specifically,the inhalation phase PH1 and the exhalation phase PH2 are detected bycalculating a maximum value B1 and a minimum value B2 of the respirationvibration value within the respiration period PC (see FIG. 5).

As illustrated in FIG. 5, one period of the respiration period(respiration cycle) PC is configured by inhalation and exhalation and isformed by inhalation executed once and exhalation executed once. In theinhalation, the diaphragm is lowered so as to draw the air into theinside thereof, and accordingly, the area of the pulmonary field in thechest increases. When the air is maximally breathed in (a transitionpoint between inhalation and exhalation), a maximum inhalation phase IMis formed. In the exhalation, as the diaphragm is raised so as tobreathe out the air, the area of the pulmonary field decreases, and,when the air is maximally breathed out (a transition point betweenexhalation and inhalation), a maximum exhalation phase EM is formed. Inthe example illustrated in FIG. 5, the respiration period PC is setbetween the maximum values B1, the respiration period PC may be setbetween minimum values PB2.

Hereinafter, the method of detecting the respiration period PC, theinhalation phase PH1, and the exhalation phase PH2 will be described.

A first method is a method of determining the respiration period PC bysequentially calculating time when the respiration vibration values area maximum value and a minimum value in the entire time of a dynamicimage and determining a maximum value B1 and a minimum value B2 of therespiration vibration value within the respiration period PC. Morespecifically, in a state in which a high-frequency noise component isdecreased by applying smoothing of the respiration vibration value inthe entire time, a maximum value (the maximum inhalation phase IM) and aminimum value (the maximum exhalation phase EM) of the respirationvibration value are calculated. Accordingly, it can be prevented that anoise component included in the respiration vibration value iserroneously detected as a maximum value or a minimum value.

A second method is a method of detecting the respiration period PC firstand detecting time when the respiration vibration values are a maximumvalue and a minimum value for every respiration period PC. A differencefrom the first method is that the maximum value (in other words, themaximum inhalation phase IM) and the minimum value (maximum exhalationphase EM) of the respiration vibration value are calculated not in theentire time but in units of the respiration periods PC. Also in thesecond method, similar to the first method, the maximum value and theminimum value may be extracted in a state in which the high-frequencynoise component is decreased by applying smoothing of the respirationvibration value.

In this way, the period classifying unit 115 sets a change of therespiration vibration value as the respiration phase PH and detects themaximum value B1 and the minimum value B2 of the respiration vibrationvalue within the respiration period PC, thereby detecting the inhalationphase PH1 and the exhalation phase PH2 (see FIG. 5).

<4-2. Frame Selecting Unit 120A>

Since the period classifying unit 115 detects the respiration period PC,the maximum value B1 and the minimum value B2 of the respirationvibration value, the inhalation phase PH1, and the exhalation phase PH2,the frame selecting unit 120A can execute the process as below.

In a case where the base frame image BF and the reference frame image RFare frame images when the respiration period PC is within the sameperiod, as the frame selecting process of (a1) described above in theframe selecting unit 120A, a first selection process and a secondselection process described below are executed.

The first selection process is the process of selecting any one frameimage from among (b1) a frame image when the respiration vibration valuecorresponds to a first set value set in advance, (b2) a frame image whenthe respiration vibration value corresponds to the maximum value B1, and(b3) a frame image when the respiration vibration value corresponds tothe minimum value B2 as the base frame image BF.

In addition, the second selection process is the process of selectingany one frame image from among (c1) a frame image when the respirationvibration value corresponds to a second set value set in advance, (c2) aframe image that is adjacent to the base frame image BF in time, (c3)when the base frame image BF is a frame image of (b2), a frame imagecorresponding to the minimum value B2 of the respiration vibrationvalue, and (c4) when the base frame image BF is a frame image of (b3), aframe image corresponding to the maximum value B1 of the respirationvibration value as the reference frame image RF.

The first set value of (b1) and the second set value of (c1) describedhere are respiration vibration values that are arbitrary designated by auser. In the first selection process, the selection process is executedusing a frame image at the time of the respiration vibration valuecorresponding to the designated first set value as the base frame imageBF. In addition, in the second selection process, the selection processis executed by using a frame image at the time of the respirationvibration value corresponding to the designated second set value as thereference frame image RF. Furthermore, as a premise of the first andsecond selection processes described above, a condition that the baseframe image BF and the reference frame image RF are limited to be withinthe inhalation phase PH1 or the exhalation phase PH2 may be applied. Inaddition, to the first and second selection processes described above, auser may arbitrary designate a base frame image BF and a reference frameimage RF for frame images within the respiration period PC.

In this way, by using the base frame image BF and the reference frameimage RF selected in the first and second selection processes executedby the frame selecting unit 120A, any of the first to fourth boundaryline extracting processes is executed, and any of the first to fourthdisplacement amount calculating process is executed (see FIG. 23).

<4-3. Basic Operation of Image Processing Device 3A>

Subsequently, FIG. 25 is a flowchart that illustrates the operation ofthe image processing device 3A according to the fourth embodiment. InFIG. 25, Steps SA1 and SA4 to SA8 are similar to Steps S1 and S3 to S7illustrated in FIG. 20, and thus, description thereof will not bepresented.

In this fourth embodiment, the period classifying unit 115 that is notprovided in the first embodiment is added, and the frame selecting means120A is substituted for the frame selecting means 120, and only thefollowing processes are changed.

As a process similar to that of the first embodiment, through Step SA1,as illustrated in FIG. 25, in Step SA2, the period classifying unit 115detects the respiration period PC of an examinee M (body) that issynchronized with the photographing time at which the plurality of frameimages MI acquired in Step SA1 and classifies the plurality of frameimages MI in units of the respiration periods PC, thereby acquiring aplurality of frame images MI′. At this time, in addition to therespiration period PC, the period classifying unit 115 simultaneouslydetects the maximum value B1 and the minimum value B2 of the respirationvibration value, the inhalation phase PH1, and the exhalation phase PH2.

In addition, in Step SA3, in consideration of the respiration period PCand the like detected in Step SA2, as the frame selecting process of(a1) described above that is executed by the frame selecting unit 120A,the first selection process and the second selection process areexecuted, whereby the base frame image BF and the reference frame imageRF are selected. The remaining process is similar to that of the firstembodiment.

As above, in the image processing device 3A according to the fourthembodiment, the base frame image BF and the reference frame image RF areframe image at the time of the respiration period PC being within thesame period, the frame selecting process of (a1) includes a firstselection process in which any one frame image of (b1) to (b3) isselected as the base frame image BF, and the second selection process inwhich any one frame image of any of (c1) to (c4) is selected as areference frame image RF. Accordingly, between frame images within thesame period, which are desired by the user, a change of the shape of thediaphragm boundary line LI can be diagnosed with high accuracy.

5. Fifth Embodiment

In a case where a follow-up observation is made by using dynamic imagesof an examinee M that are photographed in the past, it is necessary tomake a diagnosis by aligning and comparing a plurality of X-ray dynamicimages, and the comparison efficiency is low.

Thus, an object of the fifth embodiment is to acquiredisplacement-corrected boundary line information by using dynamic imagesof the examinee M of the past and the present. In this embodiment, while“present” and “past” are denoted in the head of terms, here, the meaningof denoting “present” is used as a concept of being new in time comparedwith denoting “past”.

FIG. 26 is a block diagram that illustrates the functional configurationof a control unit 31B used by an image processing device 3B configuredas a fifth embodiment of the present invention. This control unit 31B isused as a substitute for the control unit 31 (see FIG. 3) of the imageprocessing device 3 according to the first embodiment. A difference fromthe first embodiment is that, by using a dynamic image photographed inthe past and further including a period classifying unit 115B, theconstituent units are changed to a dynamic image acquiring unit 110B, aframe selecting unit 120B, a boundary line extracting unit 130B, adisplacement correcting unit 140B, and an image generating unit 150B.

In addition, as illustrated in FIG. 26, information storage device 5 isconfigured, for example, by a database server using a personal computeror a workstation, is configured to include a past image storage unit(database) 51, and transmits/receives data to/from the control unit 31Bthrough a bus 36 (see FIG. 1). In the past image storing unit 51,dynamic images of the examinee M that are photographed in the past usedfor a diagnosis are stored in advance. The remaining configuration issimilar to that of the image processing device 3.

<5-1. Dynamic Image Acquiring Unit 110B>

The dynamic image acquiring unit 110B, in addition to a present dynamicimage acquiring unit 310 that acquires a present dynamic image (aplurality of present frame images NMI) corresponding to the dynamicimage acquiring unit 110 described above, is configured to include apast dynamic image acquiring unit 210 that acquires a past dynamic image(a plurality of past frame images PMI) for the same examinee M as thatof the present frame image NMI (see FIG. 26).

The past dynamic image acquiring unit 210, for example, as illustratedin FIG. 26 acquires a past dynamic image from the past image storingunit 51. In addition, while it is desirable that the photographingranges of the present frame images NMI and the past frame images PMI arethe same, it is assumed that at least the diaphragm area is necessarilyincluded therein.

In this way, the selection target frame images TI according to the fifthembodiment are configured to include two kinds of frame images includinga present frame image NMI and a past frame image PMI photographed forthe same examinees M (body) with the photographing periods beingseparated far from each other.

<5-2. Period Classifying Unit 115B>

The period classifying unit 115B is configured to include a past periodclassifying unit 215 and a present period classifying unit 315 (see FIG.26). Here, the past period classifying unit 215 and the present periodclassifying unit 315 included in the period classifying unit 115B havethe same function as that of the period classifying unit 115 describedabove.

In other words, the present period classifying unit 315 detects apresent respiration period of the examinee M synchronized with thephotographing time at which a plurality of present frame images NMIacquired by the dynamic image acquiring unit 310 and classifies theplurality of present frame images NMI in units of the presentrespiration periods. Then, the present period classifying unit 315outputs a plurality of present frame images NMI′ after theclassification made in units of the present respiration periods to theframe selecting unit 120B (see FIG. 26).

On the other hand, the past period classifying unit 215 detects a pastrespiration period of the examinee M synchronized with the photographingtime at which a plurality of past frame images PMI acquired by thedynamic image acquiring unit 210 and classifies the plurality of pastframe images PMI in units of the past respiration periods. Then, thepast period classifying unit 215 outputs a plurality of past frameimages PMI′ after the classification made in units of the pastrespiration periods to the frame selecting unit 120B (see FIG. 26).

FIG. 27 is a schematic view that illustrates waveform data of therespiration vibration value detected by the period classifying unit 115Bin a time series. FIG. 27A is a diagram that illustrates the pastrespiration phase PPH and the past photographing timing PTM detected bythe past period classifying unit 215 altogether, and FIG. 27B is adiagram that illustrates the present respiration phase NPH and thepresent photographing timing NTM detected by the present periodclassifying unit 315 altogether.

As illustrated in FIG. 27A, the past period classifying unit 215 detectsa past respiration period PPC, a past maximum value PB1 and a pastminimum value PB2 of the past respiration vibration value, a pastinhalation phase PPH1, and a past exhalation phase PPH2. In addition, asillustrated in FIG. 27B, the present period classifying unit 315 detectsa present respiration period NPC, a present maximum value NB1 and apresent minimum value NB2 of the present respiration vibration value, apresent inhalation phase NPH1, and a present exhalation phase NPH2.

In addition, in the example illustrated in FIG. 27, while the pastrespiration period PPC (present respiration period NPC) is set betweenthe past maximum values PB1 (present maximum values NB1), the pastrespiration period may be set between past minimum values PB2 (presentminimum values NB2).

<5-3. Frame Selecting Unit 120B>

Since the period classifying unit 115B detects the past respirationperiod PPC and the like and the present respiration period NPC and thelike, the frame selecting unit 120B can execute a process as below.

The frame selecting process executed by the frame selecting unit 120Bincludes the process of selecting a past frame image PMI (a frame imagephotographed in the past) as the base frame image BF among the frameimages TI (two kinds of frame images including a present frame image NMIand a past frame image PMI).

For example, in the example illustrated in FIG. 27, in the frameselecting process executed by the frame selecting unit 120B, there areten selection target frame images TI when the selection target frameimages are configured by past frame images PT1 to PT5 and the presentframe images NT1 to NT5. Then, in the frame selecting process, a baseframe image BF is selected from among the five past frame images PT1 toPT5. Then, as the reference frame images RF, the present frame imagesNT1 to NT5 are selected.

<5-4. Boundary Line Extracting Unit 130B>

The boundary line extracting unit 130B executes a process similar tothat of the boundary line extracting unit 130 for the base frame imageBF selected by the frame selecting unit 120B and the past frame imagesPTI and the present frame images NTI corresponding to the referenceframe images RF as targets.

For example, in the example illustrated in FIG. 27, since the base frameimage BF is one of the past frame images PT1 to PT5, the boundary lineextracting unit 130B executes the process for the frame image selectedas the base frame image BF among the past frame images PT1 to PT5 andall the present frame images NT1 to NT5. In other words, the base frameimage BF is extracted from the past frame images PT1 to PT5, and thepresent diaphragm boundary lines NL1 to NL5 (NLI) are extracted from thepresent frame images NT1 to NT5 (see FIG. 26).

<5.5 Displacement Correcting Unit 140B>

The displacement correcting unit 140B executes a process that is similarto that of the displacement correcting unit 140 for the past diaphragmboundary lines PLI and the present diaphragm boundary lines NLI thatcorrespond to the base boundary line BL and the reference boundary linesRL extracted by the boundary line extracting unit 130B.

For example, in the example illustrated in FIG. 27, since the baseboundary line BL is constantly the past diaphragm boundary line PL1, andthe reference boundary line RL is one of the present diaphragm boundarylines NL1 to NL5, the displacement correcting unit 140B executes anappropriate process by using the present diaphragm boundary lines NL1 toNL5. In other words, present displacement-corrected boundary lines NL1 cto NL5 c (NLIc) can be acquired based on the present diaphragm boundarylines NL1 to NL5 by using the diaphragm boundary line PL1 as thedisplacement base (see FIG. 26).

<5-6. Image Generating Unit 150B>

The image generating unit 150B executes a process similar to that of theimage generating unit 150 for each of the past displacement-correctedboundary line PLIc and the present displacement-corrected boundary lineNLIc acquired by the displacement correcting unit 140B.

For example, in the example illustrated in FIG. 27, since the presentdisplacement-corrected boundary line NLIc is the presentdisplacement-corrected boundary lines NL1 c to NL5 c, the imagegenerating unit 150B executes the process for each of the presentdisplacement-corrected boundary lines NL1 c to NL5 c. In other words,the present displacement-corrected boundary line information NLG isacquired from the present displacement-corrected boundary lines NL1 c toNL5 c (see block diagram of FIG. 26).

<5-7. Basic Operation of Image Processing Device 3B>

Subsequently, FIG. 28 is a diagram that illustrates the operation flowof the image processing device 3B according to the fifth embodiment asan example. In this fifth embodiment, the past dynamic image acquiringunit 210, the present dynamic image acquiring unit 310, the presentperiod classifying unit 315, and the past period classifying unit 215,which are not provided in the first embodiment, and the frame selectingunit 120B replaces the frame selecting unit 120, whereby the process ischanged as below.

In other words, in Step SB1A, the past dynamic image acquiring unit 210acquires a past dynamic image (a plurality of past frame images PMI)from the past image storing unit 51 of the information storing device 5.

In Step SB2A, the past period classifying unit 215 classifies aplurality of past frame images PMI in units of the past respirationperiod PPC, thereby acquiring a plurality of past frame images PMI′. Atthis time, the past period classifying unit 215 detects the past maximumvalue PB1 and the past minimum value PB2 of the past respirationvibration value, a past inhalation phase PPH1 and a past exhalationphase PPH2 in addition to the past respiration period PPC.

In addition, in parallel with Steps SB1A and SB2A, Steps SB1B and SB2Bare executed. In other words, in Step SB1B, the present dynamic imageacquiring unit 310 acquires a present dynamic image (a plurality ofpresent frame images NMI) photographed by the reading control device 14of the photographing apparatus 1 through the photographing controldevice 2.

In Step SB2B, the present period classifying unit 315 classifies aplurality of present frame images NMI in units of present respirationperiods NPC, thereby acquiring a plurality of present frame image NMI′.At this time, the present period classifying unit 315 simultaneouslydetects the present maximum value NB1 and the present minimum value NB2of the present respiration vibration value, the present inhalation phaseNPH1, and the present exhalation phase NPH2 in addition to the presentrespiration period NPC.

In Step SB3, in the frame selecting process executed by the frameselecting unit 120B, one past frame image PMI is selected as the baseframe image BF among the selection target frame images TI (two kinds offrame images including the present frame images NMI and the past frameimages PMI). The remaining frame selecting process is similar to StepS2. In other words, in the example illustrated in FIG. 27, the pastframe image PT1 is selected as the base frame image BF, and the presentframe images NT1 to NT5 are selected as the reference frame images RF.In other words, here, the past frame images PT2 to PT5 are excluded fromthe processing target.

In Step SB4, the boundary line extracting unit 130B executes a processsimilar to that of the process of Step S3 described above for the pastframe image PTI and the present frame image NTI corresponding to thebase frame image BF and the reference frame image RF selected in StepSB3, thereby extracting the base boundary line BL and the referenceboundary line RL (present diaphragm boundary line NLI).

In Step SB5, the displacement correcting unit 140B appropriatelyexecutes a process similar to Step S4 described above for the presentdiaphragm boundary lines NLI that correspond to the base boundary lineBL and the reference boundary line RL extracted in Step SB4, therebyacquiring the present displacement-corrected boundary line NLIc.

In Step SB6, in a case where the displacement correcting unit 140Bfurther executes the process, when the diaphragm boundary line the LI(reference boundary line RL) that is the correction target is changed,the displacement correcting unit 140B instructs the frame selecting unit120B to change the reference frame image RF, and the process of StepsS3B to S5B is repeated again. On the other hand, in a case where theprocess is ended by the displacement correcting unit 140B, the processproceeds to Step SB7.

In Step SB7, the image generating unit 150B executes a process similarto Step S6 described above for the present displacement-correctedboundary line NLIc acquired in Step SB4, thereby acquiring the presentdisplacement-corrected boundary line information NLG.

In the present displacement-corrected boundary line information NLG, atleast information indicating the present displacement-corrected boundarylines NLIc is included, and it is preferable to also include informationindicating the base boundary line BL therein.

Finally, in Step SB8, the image generating unit 150B outputs the presentdisplacement-corrected boundary line information NLG generated in StepSB7 to the display unit 34 or the storage unit 32 (see FIG. 26), and theflow of this operation ends.

As above, according to the image processing device 3B according to thefifth embodiment, the selection target frame images TI include two kindsof frame images (frame images photographed in the past in time withrespect to the plurality of frame images) including the present frameimage NMI and the past frame image PMI photographed for the sameexaminee M (body) with the photographing periods thereof being separatedfar from each other, and the frame selecting process includes a processof selecting a frame image photographed for the same body in the past intime with respect to the plurality of frame images as the base frameimage. In other words, in a case where the presentdisplacement-corrected boundary line information NLG indicating thepresent displacement-corrected boundary lines NLIc is acquired, as thebase frame image BF, a common (same) frame image (the past frame imagePT1 in the example illustrated in FIG. 27) photographed in the past maybe used. Accordingly, through a dynamic-state diagnosis, a comparisonbetween the shapes of the past and the present in the diaphragm boundaryline LI of one body and a comparison between changes thereof can be madewith high accuracy. For this reason, a follow-up observation can beaccurately executed.

<5-8. First Modified Example of Fifth Embodiment: Right and Left Sidesof Diaphragm Boundary Line LI>

In the fifth embodiment, in order to display a difference between shapechanges of the past and the present of the diaphragm boundary line LI ofthe examinee M, while the present displacement-corrected boundary lineinformation NLG is acquired, for example, for the purpose of displayinga difference in shape changes of the right and left sides of thediaphragm boundary line LI in the same frame image, right-sidedisplacement-corrected boundary line information and left-sidedisplacement-corrected boundary line information may be acquired (notillustrated in the drawing).

In other words, when the displacement amount calculating process isexecuted for each of the right and left sides of the diaphragm boundaryline LI, as the base boundary line BL, a common (same) right-sidediaphragm boundary line (or the left-side diaphragm boundary line) isused. Here, it should be noted that the shapes of the diaphragm boundarylines LI of the right side and the left side have a relation of linesymmetry with respect to the spine as its symmetrical axis, and thus, byreversing the shape of anyone of the diaphragm boundary lines LI of theright and left sides, the displacement amount calculating process isexecuted, and the right-side displacement-corrected boundary line andthe left-side displacement-corrected boundary line need to be acquired.

<5-9. Second Modified Example of Fifth Embodiment: Inhalation Phase PH1and Exhalation Phase PH2>

In a second modified example, for the purpose of displaying a differencein the shape changes at the time of the inhalation phase PH1 of thediaphragm boundary line LI and at the time of the exhalation phase PH2,inhalation displacement-corrected boundary line information andexhalation displacement-corrected boundary line information may beacquired (not illustrated in the drawing).

In other words, when the displacement amount calculating process isexecuted at the time of the inhalation phase PH1 of the diaphragmboundary line LI and at the time of the exhalation phase PH2, as thebase boundary line BL, a common (same) inhalation diaphragm boundaryline (or an exhalation diaphragm boundary line) is used, and aninhalation displacement-corrected boundary line and a left-sidedisplacement-corrected boundary line are acquired.

6. Sixth Embodiment

As it is difficult to perceive the motion of the diaphragm area directlyfrom the diaphragm boundary line LI as described above, it is alsodifficult to perceive the motion of the heart directly from the heartboundary line. In other words, since a motion other than the actualmotion of the heart is accompanied, it is difficult to preciselydiagnose the motion of the heart.

Thus, while the first to fifth embodiments relate to a case where thetarget area is the diaphragm area, in a sixth embodiment, a case will beconsidered in which the target area is the heart area. A difference fromthe first embodiment is that the boundary line extracting unit 130 isreplaced with a boundary line extracting unit 130C that extracts theboundary line of a heart area. In addition, as illustrated in the fourthand fifth embodiments, in a case where the period classifying unit isincluded, it is replaced with a period classifying unit 115C thatclassifies a plurality of frame images MI based on a periodical changeof the heart. The remaining configuration is similar to that of theimage processing device according to one of the first to fifthembodiments.

Like this embodiment, in a case where the target area is the heart area,in addition to the motion of the heart as the motion of the heart at thetime of the presence of respiration, the motion of the respiration isaccompanied. Thus, even when a correction process is executed throughthe first displacement amount calculating process in which only thevertical motion is considered, an appropriate displacement-correctedboundary line LIc cannot be acquired. For this reason, a displacementamount D is preferably calculated through the second to fourthdisplacement amount calculating processes. Alternatively, a method maybe used in which, after the displacement amount D of the diaphragm areais removed, the displacement amount D of the heart area is calculatedthrough the first displacement amount calculating process.

Hereinafter, first, the boundary line extracting unit 130C will bedescribed, and then, the period classifying unit 115C will be described.

<6-1. Boundary Line Extracting Unit 130C>

The boundary line extracting unit 130C executes a boundary lineextracting process in which a first number of heart boundary lines(target area boundary lines) are acquired by extracting the boundarylines of the heart area for the first number of frame images TI.

As a technique for detecting the heart boundary line (the contour of theheart) from each frame image, various known techniques can be employed,and, for example, a technique for detecting the contour of the heart bymatching characteristic points of an X-ray image and characteristics ofa hear model by using a model (heart model) representing the shape ofthe heart (for example, see “Image feature analysis and computer-aideddiagnosis in digital radiography: Automated analysis of sizes of heartand lung in chest images”, Nobuyuki Nakamori et al., Medical Physics,Volume 17, Issue 3, May, 1990, pp. 342-350) or the like may be employed.

The method of extracting the heart boundary line HLI is not limited tothe methods described above, but any other method capable of extractingthe heart boundary lines from a dynamic image may be used.

FIG. 29 is a schematic view that illustrates heart boundary linesextracted from each frame image. As illustrated in FIGS. 29A to 29C, itcan be understood that heart boundary lines HL1 to HL3 (HLI) areextracted based on each frame image.

<6-2. Period Classifying Unit 115C>

The period classifying unit 115C detects a so-called cardiac cycle(target area period) in which the heart area of an examinee M (body)synchronized with the photographing time, at which a plurality of frameimages MI are photographed by the dynamic image acquiring unit 110,periodically changes and classifies the plurality of frame images MI inunits of the cardiac cycles. Then, the period classifying unit 115Coutputs a plurality of frame images MI′ after the classification inunits of the cardiac cycles to the frame selecting unit.

Hereinafter, a cardiac cycle acquiring process for detecting the cardiaccycle of the examinee M that is executed by the period classifying unit115C will be described.

<6-2-1 Cardiac Cycle Acquiring Process>

The cardiac cycle acquiring process is a process for acquiring a heartcycle by calculating the motion amount of the heart wall (in otherwords, it corresponds to the heart boundary line HLI) by usingphotographed images acquired by the dynamic image acquiring unit 110.Described in more detail, as a variation in the heart wall is detectedfrom the dynamic image, the phase of the heartbeat is detected at thetiming at which each frame image is photographed. Then, the cardiaccycle is determined based on the phase of the heartbeat.

As illustrated in FIG. 29, as an example of the variation of the heartwall (heart boundary line HLI) perceived from the dynamic image, avariation in the horizontal width of the heart will be employed. Inother words, in FIGS. 29A to 29C, in the process of expansion of theheart, a state is illustrated as an example in which the horizontalwidth of the heart increases from w1 to w3.

Thus, from each frame image, by using the method described above or anyother method, the contour (heart boundary line HLI) of the heart isdetected, and, by detecting the horizontal width of the heart, thecardiac cycle can be detected.

FIG. 30 is an exemplary schematic view that illustrates a relationbetween photographing time and the horizontal width (the motion amountof the heart wall) of the heart in a plurality of frame imagesconfiguring a dynamic image. In FIG. 30, the horizontal axis representsthe time, the vertical axis represents the horizontal width of theheart, and each mark represents the value of the detected horizontalwidth of the heart.

Here, when the horizontal width of the heart acquired at time t is Hwt,and the horizontal width of the heart acquired at time t+1 is Hwt+1,and, in a case where (Hwt+1−Hwt)≧0 is satisfied, a frame image perceivedat time t is classified into the expanding time of the heart, and, in acase where (Hwt+1−Hwt)<0 is satisfied, a frame image perceived at time tis classified into the shrinking time of the heart.

In this way, by detecting the horizontal width of the heart, in otherwords, variations in the heart wall, the time can be classified into theexpanding time of the heart or the shrinking time of the heart, andaccordingly, the phase of the heart beat can be detected.

As above, the period classifying unit 115C detects the cardiac cyclebased on the motion of the heart wall perceived from the dynamic image,whereby a plurality of frame images MI can be classified in units ofcardiac cycles.

The cardiac cycle acquiring process may be executed not only by usingthe method described above but also a method acquiring the cardiac cycleusing a result acquired by an electrocardiograph. For example, such amethod may be realized by executing a detection operation executed by aphase detecting unit of the electrocardiograph to be synchronized withthe imaging operation of the photographing apparatus 1.

As above, the image processing device according to the sixth embodimenthas the heart area as the target area and can appropriately diagnose adisease relating to the heart through a dynamic state diagnosis. In thecase of a minor symptom, while the abnormality may not be noticed, bymaking a diagnosis using the displacement-corrected boundary lineinformation LG, the diagnosis does not depend on the user'ssubjectivity, whereby an erroneous diagnosis can be prevented.

7. Modified Example

As above, while the embodiments of the present invention have beendescribed, the present invention is not limited to the embodimentsdescribed above but various changes may be made therein.

Here, while the embodiments are separately illustrated such that theimage processing devices 3, 3A, 3B, and the like are individuallyexecuted, such individual functions may be combined together as long asthere is no contradiction.

In the display unit 34 of this embodiment, while the display methodillustrated in FIG. 18 has been illustrated, the display method is notlimited thereto. For example, the displacement-corrected boundary lineinformation LG of an examinee M determined to be abnormal may bedisplayed on the dynamic image in an overlapping manner such that adisplayed line of the actual dynamic image is indicated. In addition,the displacement-corrected boundary line information LG displayed on thedynamic image may be additionally displayed not only in a case where anabnormality is determined but also before/after the maximum exhalation,the maximum inhalation, or a periodical position at which an occurrenceof a disease is suspicious. Furthermore, as illustrated in FIG. 31, whena moving image is reproduced, it may be configured such that thedisplacement-corrected boundary line L5 c corresponding to the maximumexhalation and the displacement-corrected boundary line L1 ccorresponding to the maximum inhalation are constantly displayed, andthe other displacement-corrected boundary lines are not displayed.

While the image processing device 3 according to the first embodimenthas been described to be configured to include the frame selecting unit120, the present invention is not limited thereto, but a configurationnot including the frame selecting unit 120 may be employed. In otherwords, in the configuration not including the frame selecting unit 120,the boundary line extracting unit 130 executes the boundary lineextracting process for all of the plurality of frame images MIconfiguring a dynamic image acquired by the dynamic image acquiring unit110. Then, the displacement correcting unit 140, under a condition (apredetermined rule) designated by the user in advance, sequentially setsa base boundary line BL and reference boundary lines RL, whereby thedisplacement amount calculating process and the correction process areexecuted.

In this embodiment, while the frame selecting process executed by theframe selecting unit 120 is executed as a pre-process of the boundaryline extracting process executed by the boundary line extracting unit130, the present invention is not limited thereto, but the frameselecting process may be configured as a post-process of the boundaryline extracting process. In other words, after the boundary lineextracting process is executed for all the plurality of frame image MIconfiguring the dynamic image acquired by the dynamic image acquiringunit 110, in the frame selecting process, the base frame image BF (baseboundary line BL) and the reference frame image RF (reference boundaryline RL) are selected. In other words, simultaneously with the executionof the frame selecting process, the diaphragm boundary line LI of thebase frame image BF is set as the base boundary line BL, and thediaphragm boundary line LI of the reference frame image RF is set as thereference boundary line RL.

While each image processing device according to this embodiment handlescases where the target area is the diaphragm or the heart area, thetarget area is not limited thereto, and the target area may be thediaphragm area and the heart area. In other words, the process until thedisplacement-corrected boundary line information LG is acquired isexecuted parallel for the diaphragm area and the heart area, and thedisplacement-corrected boundary line information LG is separately outputto the display unit 34 and the storage unit 32 for each target area.

In the first modified example of the fifth embodiment, while the purposeis regarded to be the acquisition of the right-sidedisplacement-corrected boundary line and the left-sidedisplacement-corrected boundary line, the purpose is not limitedthereto. Thus, for example, for diaphragm boundary lines LI of healthypersons and diaphragm boundary lines LI of unhealthy persons, byexecuting the process by using a common (same) healthy person diaphragmboundary line as the base boundary line BL, a displacement-correctedboundary line of healthy persons and a displacement-corrected boundaryline of unhealthy persons may be acquired.

Here, the subject is not limited to a human body but may be an animalbody.

While the present invention has been described in detail, thedescription presented above is an example in every aspect, and thepresent invention is not limited thereto. In addition, it is understoodthat unlimited number of modified examples not illustrated here may beconsidered without departing from the scope of the present invention.

REFERENCE SIGNS LIST

-   1 Photographing apparatus-   2 Photographing control device-   3, 3A, 3B Image processing device-   31, 31A, 31B Control unit-   Storage unit-   Display unit-   100 Radiation dynamic image photographing system-   110, 110B Dynamic image acquiring unit-   115, 115B Period classifying unit-   120, 120A, 120B Frame selecting unit-   130, 130B Boundary line Extracting unit-   140, 140B Displacement Correcting unit-   150, 150B Image Generating unit-   210 Past dynamic image acquiring unit-   215 Past period classifying unit-   310 Present dynamic image acquiring unit-   315 Present period classifying unit-   M Subject (examinee)-   MI Frame image-   TI First number of frame images-   BF Base frame image-   RF Reference frame image-   BL Base boundary line-   RL Reference boundary line-   LI Diaphragm boundary line-   HLI Heart boundary line-   LIc Displacement-corrected boundary line-   LG Displacement-corrected boundary line information-   PC Respiration Period-   PH Respiration Phase-   PH1 Inhalation phase-   PH2 Exhalation phase-   EM Maximum exhalation phase-   IM Maximum inhalation phase

The invention claimed is:
 1. An image processing device comprising: adynamic image acquiring means that acquires a dynamic image configuredby a plurality of frame images acquired by sequentially photographing atime-varying physical state of a target area inside a human body or ananimal body in a time direction; a boundary line extracting means thatexecutes a boundary line extracting process in which a plurality oftarget area boundary lines are acquired by extracting boundary lines ofthe target area for a plurality of frame images among the plurality offrame images; a displacement correcting means that acquires apredetermined number of displacement-corrected boundary lines in which aremoval-required component is removed by executing a displacement amountcalculating process in which a displacement amount, which is theremoval-required component, is calculated using a base boundary line asa displacement base for one or more of target area boundary lines otherthan the base boundary line among the plurality of target area boundarylines by using pixels corresponding to the plurality of target areaboundary lines and executing a correction process in which apredetermined number of the target area boundary lines other than thebase boundary line are corrected by using the displacement amount afterthe displacement amount calculating process; and a display means thatdisplays displacement-corrected boundary line information for displaybased on the predetermined number of displacement-corrected boundarylines.
 2. The image processing device according to claim 1, wherein theremoval-required component includes at least one component amongdeformation components according to a vertical motion, a parallelmotion, and rotation in the target area.
 3. The image processing deviceaccording to claim 1, further comprising a frame selecting means thatexecutes a frame selecting process including a process of selecting abase frame image used for extracting the base boundary line and areference frame image used for extracting the target area boundary linesother than the base boundary line for selection target frame imagesincluding at least the plurality of frame images, wherein thedisplacement amount calculating process includes a process ofcalculating a displacement amount between corresponding pixels of thetarget area boundary line of the base frame image as the base boundaryline and the target area boundary line of the reference frame image. 4.The image processing device according to claim 3, wherein the selectiontarget frame images include frame images photographed in the past intime with respect to the plurality of frame images, and the frameselecting process includes a process of selecting a frame imagephotographed for the same body in the past in time with respect to theplurality of frame images as the base frame image.
 5. The imageprocessing device according to claim 3, further comprising a periodclassifying means that detects a target area period in which aperiodical change of the target area of the body synchronized withphotographing time at which the plurality of frame images arephotographed occurs and classifies the plurality of frame images inunits of the target area periods, wherein the base frame image and thereference frame image are frame images when the target area periods arewithin a same period, a value representing a time-varying physical stateof the target area is defined as a physical state value, and the frameselecting process includes a first selection process selecting one frameimage as the base frame image from among (b1) a frame image when thephysical state value corresponds to a first set value set in advance,(b2) a frame image when the physical state value corresponds to amaximum value, and (b3) a frame image when the physical state valuecorresponds to a minimum value, and a second selection process selectingone frame image as the reference frame image from among (c1) a frameimage when the physical state value corresponds to a second set valueset in advance, (c2) a frame image that is adjacent to the base frameimage in time, (c3) a frame image corresponding to the minimum value ofthe physical state value when the base frame image is the frame image of(b2), and (c4) a frame image corresponding to the maximum value of thephysical state value when the base frame image is the frame image of(b3).
 6. The image processing device according to claim 3, wherein thedisplacement amount used for the correction process is a displacementamount between corresponding pixels of the base boundary line and one ofthe target area boundary lines, and the target area boundary lines otherthan one of the target area boundary lines are corrected using thedisplacement amount.
 7. The image processing device according to claim3, wherein the displacement amount used for the correction process is adisplacement amount from the target area boundary line nearest in timefrom the target area boundary line that is a correction target.
 8. Theimage processing device according to claim 3, wherein the displacementamount used for the correction process is a displacement amount betweenthe base boundary line and the target area boundary line, which is thecorrection target, that is acquired as a sum of displacement amounts oftwo boundary lines adjacent in time.
 9. The image processing deviceaccording to claim 1, further comprising an image generating means thatgenerates a predetermined number of separate images separated for eachpredetermined number of displacement-corrected boundary lines, whereinthe display means sequentially displays the predetermined number ofseparate images as displacement-corrected boundary line information. 10.The image processing device according to claim 1, further comprising animage generating means that generates one still image such that theplurality of displacement-corrected boundary lines are displayed in anoverlapping manner, wherein the display means displays the still imageas the displacement-corrected boundary line information.
 11. The imageprocessing device according to claim 1, wherein the target area includesat least one of a diaphragm area and a heart area.
 12. A non-transitoryrecording medium storing a computer readable program that causes acomputer to serve as the image processing device according to claim 1 bybeing executed by the computer included in the image processing device.13. The image processing device according to claim 2, further comprisinga frame selecting means that executes a frame selecting processincluding a process of selecting a base frame image used for extractingthe base boundary line and a reference frame image used for extractingthe target area boundary lines other than the base boundary line forselection target frame images including at least the plurality of frameimages, wherein the displacement amount calculating process includes aprocess of calculating a displacement amount between correspondingpixels of the target area boundary line of the base frame image as thebase boundary line and the target area boundary line of the referenceframe image.
 14. The image processing device according to claim 2,further comprising an image generating means that generates apredetermined number of separate images separated for each predeterminednumber of displacement-corrected boundary lines, wherein the displaymeans sequentially displays the predetermined number of separate imagesas displacement-corrected boundary line information.
 15. The imageprocessing device according to claim 2, further comprising an imagegenerating means that generates one still image such that the pluralityof displacement-corrected boundary lines are displayed in an overlappingmanner, wherein the display means displays the still image as thedisplacement-corrected boundary line information.
 16. The imageprocessing device according to claim 2, wherein the target area includesat least one of a diaphragm area and a heart area.
 17. A non-transitoryrecording medium storing a computer readable program that causes acomputer to serve as the image processing device according to claim 2 bybeing executed by the computer included in the image processing device.18. The image processing device according to claim 3, further comprisingan image generating means that generates a predetermined number ofseparate images separated for each predetermined number ofdisplacement-corrected boundary lines, wherein the display meanssequentially displays the predetermined number of separate images asdisplacement-corrected boundary line information.
 19. The imageprocessing device according to claim 3, further comprising an imagegenerating means that generates one still image such that the pluralityof displacement-corrected boundary lines are displayed in an overlappingmanner, wherein the display means displays the still image as thedisplacement-corrected boundary line information.
 20. The imageprocessing device according to claim 3, wherein the target area includesat least one of a diaphragm area and a heart area.